The required maintenance under normal operating conditions is pretty minimal. The engine oil and filter has to be changed every 5K miles;Toyota dealers in my area do it for $20 - 25 (matching the price of the local pump & dump oil changers). You can do it yourself and save a few more $ if you are slightly mechanically inclined. Nowadays, the biggest problem is getting rid of the waste oil. The other required maintenance is to visually check the brake linings / pads and to rotate the tires (if you care enough to do this - I don't).
Your dealer is using scare tactics to encourage you to have him maintain the vehicle and so that he can sell you unneeded services. Typical but very unprofessional IMHO. I always tell the dealer to just do the minimum maintenance schedule.
Here is what Toyota's Maintenance guide for 2004 vehicles has to say about this subject:
http://www.toyotapartsandservice.com/smg/pdfs/auto_service_04.pdf
Maintaining your vehicle according to the recommendations in this booklet is required to ensure that your warranty coverage remains intact. You should keep detailed records of vehicle maintenance, including date of service, mileage at time of service and a description of service and/or parts installation performed.
Toyota will not deny a warranty claim solely because you do not have records to show that you maintained your vehicle. However, damage or failures caused by lack of proper maintenance are not covered under warranty.
Maintenance Providers
Maintenance and repair services may be performed by you or by any automotive service provider you choose. Toyota will not deny a warranty claim solely because you used a service provider other than a Toyota dealership for maintenance and repairs. However, damage or failures caused by improper maintenance or repairs are not covered under warranty.
Dealer-Recommended Maintenance
Your dealer may recommend more frequent maintenance intervals or more maintenance services than those listed in the scheduled maintenance log. These additional services are not required to maintain your warranty coverage. Ask your dealer for an explanation of any recommended maintenance not included in the scheduled maintenance log.