How can I lose weight on a budgent?

Good calories + move. There's a good app called "lose it!" that helps you determine your daily caloric requirements based on how much & how quickly you want to lose. It takes into account your exercise too. You really have to move in order to lose, so even just walking every day is good & costs you nothing. A low glycemic diet will give you the best quality calories. Organic is best if you can afford it.
Exercise. Pick fruits an vegetables that aren't as expensive, but avoid the simple carbohydrates (chip, crackers, white rice, etc). Also, exercise can be free - simply getting out and walking vigorously will help.