I love a beach town in the off season. You can't swim, but otherwise you get all the charms with none of the crowds. The "shoulder season," when it's too cold for the masses but just barely warm enough to wade on a nice day, is truly the best.
Rehoboth is the closest major beach to DC and Baltimore, an easy two-hour drive. It's widely-known as a gay retreat, and is a little more middle-aged oriented than nearby kid-oriented Bethany Beach or college party-oriented Ocean City, MD.
As with so many beach towns, it's got some charming urbanism, anchored by a short beachfront boardwalk, and then a main street jutting out perpendicularly from the beach.
Several of the alleys off the main street have been converted to pedestrian shopping streets. They're adorable and great.
About 5 miles inland there's a nice bus station. Bus service is good during the summer, not great the rest of the time. There are nice 24/7 bus lanes though.
Halfway between Rehoboth and Bethany lies the Charles Cullen Bridge, a big cabled-stayed deal that crosses an inlet. At this point the beach is part of a Delaware state park, so there's no boadwalk, shops, or hotels. I can't think of another beach with a bridge like this right behind it.
That's all!