Now, the above may not be a retro console per se, but it does feature some classic Atari tech despite the admittedly contemporary Nintendo DS screen. Other than that, it's an excellent and fully working Atari 2600 (VCS) handheld created from scratch. Have a look (and do also give this link a try).
Related posts: SEGA Game Gear, Atari Lynx Commercials, the Dreamcast VMU
Related Tags: Atari 2600, Atari VCS, Atari, Nintendo Ds, mod, portable 2600, gaming, games, retro