OK, from what I understand, you need an Aux-Box (or equivalent, I got an Aux-Pod, which is the same thing, but it allows control of the iPod). Look here:
http://www.pac-audio.com/
It's basically what you described...except it plugs into where a CD Changer would (at least on the back of my NAV). When it is in use, it says "CD Changer" on the screen. I did have to get a part from Toyota though, because the plug/harness is a different shape. (I think I wrote it somewhere in the thread).
Also, check this page out:
http://store.yahoo.com/logjam/auxtoy.html
Looks like it has multiple solutions to what you're looking for. I think they should work when you get the Toyota part. This place also sells the same thing, except they call it the "TOY03-F/MM Toyota "Y" Harness". I have confirmation that they are indeed the same parts... except the Toyota dealership only charged me $15.