If a breast lump is irregular instead of round, is it more likely to be cancer?

Yes. In general, cancers tend to grow in an irregular, crab-like pattern (crab in latin is cancer). However, many breast cancers can feel round and smooth. Bottom line: any new lump in the breast should be evaluated by a physician, regardless of the shape.
Not necessarily. Biopsy (a sample of the tissue in question usually done with a needle as an out patient) is the only definitive way to find out whether a non cystic lump is cancer, although there are certain characteristics thatt may allow your doctor to watch a mass , usually at 6 month intervals, to determine if it has a worrisome growth pattern.