Some heuristics has a strong underlying theory; They are either derived in a top-down manner from the theory or are arrived at based on either experimental or real world data. Others are just rules of thumb based on real-world observation or experience without even a glimpse of theory. The latter are exposed to a larger number of pitfalls.
When a heuristic is reused in various contexts because it has been seen to “work” in one context, without having been mathematically proven to meet a given set of requirements, it is possible that the current data set does not necessarily represent future data sets ( see: overfitting) and that purported “solutions” turn out to be akin to noise.
See more at: Algorithm heuristic machine learning shirt