What is the difference between following a site/person and favorite a site/person?

When you click on the "Follow" button in a site, any new site content will appear on your home dashboard "Latest" tiles. You do not have to go to the sites one by one to see the new content.

When someone posts on the site feed, those feed posts will also appear on your home feed.

If you click on the star next to a content/people/site, it means you have favorited that content/people/site for easy navigation. You can find all your favorites by clicking on your image at the upper right, then go to "Favorites".