It's all relative to tuning virtual memory to avoid having frames being swapped out to the page file.
Linux -> Create no swap partition. Windows -> Go into system properties and disable system paging file. You can disable it on both.
www.fit-pc2.com Great devices. I own one and it's perfect. Not as small as a Shiva plug, but it has more features and is x86 compatible.
Unless you're talking about SSD, you don't want a swap file on flash. Not so much for reliability these days--USB flash sticks DO have wear leveling and can remap dead sectors, but because they are horribly slow to write to, typically <10MB/s. SSD write speeds are only acceptable because the flash is arranged with a VERY wide bus to increase throughput. I say disable the swap if you have 2GB of RAM. It's hardly necessary for a server anyway since most of the heap won't be tied up on internet browser crashes (assuming you aren't using it as a general computing device as well).
You can even get away with a lot less than 2GB, especially if you think carefully about what you want to run (like choosing fluxbox over KDE/Gnome, or lighttpd over Apache). On my laptop I rarely exceed 800MB, and the embedded system I use for a server is using only 10MB (out of its 32MB total) right now, with HTTP and FTP servers running, me being logged in via SSH, and a torrent in the background (have to admit 32MB can be a tight squeeze though, so I added 60MB of swap for unexpected peaks). Long story short, think about what you need instead of installing random crap, and unless maybe you insist on some big database you shouldn't really need more than 512MB-1GB of RAM, and definitely no swap.