Weather affects our mood and behavior, and through them, many aspects of our life. When it is sunny, people become happier and smile, but when it rains, some get depressed. Despite this evidence and the abundance of weather data, weather has mostly been overlooked in the machine learning and data science research. This work shows how causal analysis can be applied to discover the effects of weather on TV watching patterns and how it can be applied for user modeling. We make several contributions. First, we show that some weather attributes, e.g., pressure and precipitation, cause significant changes in TV watching patterns. Second, we compare the results obtained for different levels of user granularity and different types of users. This showcases that causal analysis can be a valuable tool in user modeling. To the best of our knowledge, this is the first large-scale causal study of the impact of weather on TV watching patterns.