As Charlie prepares to leave for lunch, his light-colored vest is visible beneath his coat. As he exits the building, his vest is now much darker, and blends in with his coat and trousers.
When Charlie goes to lunch, he changes out of his waiter's coat and tie into his "Little Tramp" coat and vest. When he buttons his vest, the buttons and button holes are out of alignment. However, when he walks through the restaurant and onto the sidewalk, the buttons and button holes are properly aligned.