Quite unexpectedly, kinetic theory is found to specify the correct definition of average value to be employed in nonextensive statistical mechanics. It is shown that the normal average is consistent with the generalized Stosszahlansatz (i.e., molecular chaos hypothesis) and the associated $H$ theorem, whereas the $q$ average widely used in the relevant literature is not. In the course of the analysis, the distributions with finite cutoff factors are rigorously treated. Accordingly, the formulation of nonextensive statistical mechanics is amended based on the normal average. In addition, the Shore-Johnson theorem, which supports the use of the $q$ average, is carefully re-examined and it is found that one of the axioms may not be appropriate for systems to be treated within the framework of nonextensive statistical mechanics.

• Received 4 December 2008

DOI:https://doi.org/10.1103/PhysRevE.79.041116