The first thing that comes to mind while talking about football and its alienation in the US is the use of the term "soccer". While the rest of the world calls it "football", America distinguishes itself by terming it as "soccer".
While soccer gained increased popularity in the late 19th and early 20th century throughout the world, America was reluctant to adapt the game and imbibe it in their culture.
In every culture, there is a small reservation for sporting activities. Once this place has been filled, it's next to impossible to allow space for other sports to grow at the same pace simultaneously.
As specific sports get imbibed in conscience as a 'national pastime' it is hard to replace that.

In the US, this is applicable for baseball, American football and Basketball; the three sports they win every Gold Medal in the Olympics. When it comes to soccer, the best performance of the US was in the inaugural edition of the World Cup when they won the third place. The teams there were 'invited' to play and didn't have to qualify to play like modern times.

The most important factor is perhaps the Americans' will to be distinguished from the English after the American Revolution. Football was born and bred in English and spread to countries with England at the centre. The US chose to distinguish itself as a unique country and sports play quite a crucial role in that.