Refined information retrieval and frequency distribution

This paper presents the process of refining the document and their terms in Information Retrieval. It also shows the significance of this process prior to applying any of the information retrieval applications including probability models on the actual terms distribution. This is an important issue in language models approach, it also helps to show the effectiveness and efficiency in terms of minimizing the amount of time and space required to process the data. This is also very important for probabilistic approaches such as Single Poisson, double Poisson, Binomial and Multinomial distributions which are used to define the weights in the document matching process. This approach is applied on specific data sources rather than Web pages.