Rufus tells he was bullied when he lived in Chicago by a guy named Tannen. This is a homage to the bully Biff Tannen from another time travel movie, Back to the Future (1985).
The Federal military uniform jackets and trousers that Rufus and Wyatt wear indicate artillery due to the red NCO rank chevrons on the jacket sleeves with red NCO striping on the trousers legs. However, their caps have the infantry bugle on them; for artillery, the cap device should be crossed cannon to indicate artillery. For infantry the chevrons would be sky blue while the trouser striping would be in navy blue. Soldiers, especially Federal troops, would not mix and match uniform pieces. This infraction would have been immediately noticed since they were in Washington which was crawling with officers and soldiers.
Lucy bought a dress in this episode. Most ladies' clothing was not bought off the rack until much later. Women went to seamstresses to have clothing made by hand, especially formal evening wear. Lucy's dress fit way too perfectly for a 'ready made' item.
In episode 2 Abraham Lincoln is shown to have seen the end of the American Civil War. To the contrary Lincoln died April 15th 1865, while the civil war ended May 9th 1865. Lincoln was shot 24 days before the end of the civil war
Rufus tells the other soldiers that his name is Denzel Washington. This is most likely as tip of the hat to Denzel Washington's portrayal of Private Trip in the movie "Glory," which is a film that follows the U.S. Civil War's first all-black volunteer company, fighting prejudices from both his own Union Army, and the Confederates.
At the end of the episode, the song Mad World can be heard. The same song was used at the end of the movie Donnie Darko, also a time travel story.