Reliability and Validity of the Empirical Research

Question: Discuss about theReliability and Validity of the Empirical Research. Answer: Introduction Reliability as well as validity are two significant aspects for approving and validating the quantitative research. Reliability is a measure of the degree to which there is consistency in the results when the repetition of the experiment is done several times under the methodological conditions that remain the same (Joppe, 2000). On the other hand, validity may be defined as the degree it is supported by the evidence that the data interpretations are correct and the manner of the use of interpretations are also appropriate (Moskal et al., 2002). It can also be said that validity is said to be present in a research in case the results that are obtained are believable as well as truthful. For determining validity, a number of questions are posed by the researchers and mostly they look up the research that was conducted by other people to answer the question regarding the accuracy of the measurements (Joppe, 2000). This paper will be analysing in a careful manner the article The impact of brand gender on brand equity Findings from a large-scale cross-cultural study in ten countries (Lieven Hildebrand, 2015). A discussion as well as an analysis of the reliability and also the validity of the empirical research will be carried out. It will also include the instrument wherever it is applicable. In case there are any possible flaws in the reliability and validity, they will be pointed out. Suggestions will also be made about the ways in which the researcher might have conducted tests for both or either reliability and validity. Discussion and Analysis of Reliability The meaning of reliability is that the scores which an instrument provides are consistent as well as stable (Creswell, 2005). Reliability may be defined as the consistency of the measurement or the extent to which measurement can be done by the instrument in the same way every time its use is made under similar conditions and with subjects who are similar. It can be considered the repeatability of measurement. If the score of a person on the same test which is conducted twice is similar both the times, then the measure can be called reliable. However, it is to be kept in mind that an estimation of reliability is done and not its measurement (Kirk Miller, 1986). Three kinds of reliability were identified in quantitative research. These are in relation to production of same results under the conditions of measurement that remain the same, the measurement stability with respect to time and similarity of measurements within a given period of time. Reliability is the consistency with whi ch the scores of an individual stay relatively similar and the test-retest methods can be used for determining them at different times. Such an instrument is a stable instrument. If stability is of high degree it is an indication that reliability is of high degree showing that the results can be repeated. Reliability is mostly estimated in two ways either by means of test/retest or through internal consistency. Test / Retest Test / Retest - It is a comparatively more conservative method for estimating reliability. The idea that lies behind is that the same score should be obtained by you on both Test 1 as well as Test 2. The key components of such a method are implementing the instrument of measurement two times separately for each subject, computing the correlation that exists between two distinct measurements and assuming that the underlying condition (or the trait which is to be measured) does not involve any changes and is same for both the tests. Internal Consistency The estimation of reliability is done through internal consistency by forming the groups of questions measuring the same concept in a questionnaire. For instance, two sets can be formed of three questions measuring the same concept such as class participation and subsequent to the collection of the responses, a correlation is run between these two groups having three questions each for determining if the concept is being measured by the instrument in a reliable way. Cronbachs Alpha is a common way for the computation of correlation values among the questions that are present on your instrument. Each question on the questionnaire is split in all possible ways and the correlation value for all of them is computed using computers. The closer the value is to one, the higher will be the instruments reliability estimate. In the study, for testing of the hypotheses, a total number of 3049 consumers were selected through a professional agency of market research in nations spanning across four continents. In all the nations, the participants were presented with 20 brands in total in eight categories of products when the study began. A careful selection was also made for the brands involved in this study and it was conditional based on their presence in every selected nation. Thus, in this study a very large number of participants as well as a large number of countries are involved and hence carrying out a retest in such cases for testing the reliability of the instrument will not be possible. For ensuring the quality of data for the study and for preventing consumer ratings for brands that they did not know, the participants had to choose the brands regarding which they have awareness. A choice of at least one brand was necessary for participating in the study (Dolnicar Rossiter, 2008). The rating of t hese brands was done along the brand gender as well as brand equity that was perceived by them. There is a high probability that consumers might choose some other brand in the next study or their perceptions might differ for even the same brands and hence it will affect the reliability. Additionally the ratings resulted in overall 5.6 brands being rated by each participant and 16,934 cross-clustered observations were a part of the study. Collection of such large scale data for retesting is not practical as it will be involving large expenses. The differences in the languages between nations and also the social norms in those countries might be affected by the results. Internal validity is also lacking as most of the values are not very close to one which is gives the reliability estimate with respect to the instrument. Discussion and Analysis of Validity Validity is described as the extent to which a quantitative study is able to measure a concept accurately. There are a number of threats to validity that either help in proving or raising issues related to the accuracy of the data or application or results or statistical tests application for concluding the effects that an outcome has (Creswell, 2003). The various types of threats to validity consist of the external threats, internal threats, construct validity tests as well as statistical conclusion threats. The basic requirement for interpreting an experiment is to define internal validity in a clear way (Campbell Stanley, 1963). The threats of external validity crop up when incorrect inferences from the same data are concluded by the researcher to the other persons. The question of generalisability is addressed by it that is to whom the results obtained can be generalised. The threats of internal validity are procedures of the experiments, treatments or the participants experiences which pose a threat to the ability of the researcher of drawing correct inferences from the data that is present in the experiment. These arise because of inadequate procedures are used such as change of the tool or instrument during the course of the experiment, changes in the participants of the control group under the study, etc. due the these procedures that are inadequate, the experimenter needs to figure out whether the experiments are able to create any differences in such an instance or not. Construct validity threat takes place when inadequate definitions are used by the investigators and the variables are measured on the basis of the definitions which are inadequate. The threat if statistical construction validity takes place when inaccurate inferences are drawn by the experimenters from the data due to the statistical test (that is used for the collected data) assumptions being violated. Generally, the methods that are used for the establishment of validity in case of quantitative research consist of data triangulation, experiment review, regression analysis, participant feedback and statistical analysis. In the study, for ensuring the content validity, all the surveys that were conducted across the different countries were identical and online method was used. In cases of USA, India as well as Australia, the original versions in English was used while for seven other nations, use of the professional services of language editing was made for the translation of the survey. This increased the chances of measuring the concepts in an accurate way. Construct validity is present in the data as adequate definitions have been provided by the researcher for all the constructs that are to be measured. However, collection of the data from such a large number of participants across nations required a lot of analysis which increased the probability of incorrect inferences being drawn by the researcher. The inferences are based on the perceptions of the customers with regards to the various brands and interpreting them in an accurate way is difficult. Suggestions for Enhancing and/or Testing for Either or Both of Reliability or Validity The researcher could have enhanced and tested for both the reliability as well as the validity of the research. Reliability could have been enhanced by conducting separate researches for countries that have a collectivist culture and for countries having an individualistic culture. This would have made it easier to compare and contrast the responses of the people from the two types of nations. Alternatively, the researcher could also have taken a selected few nations and then conducted the research instead of taking so many nations at one go for the purpose of research because this only helped in further complicating the matter. Apart from this, it would have been easier to conduct a re-test on people belonging to a particular type of culture which would have given accurate estimates of reliability. Similarly, for enhancing the validity also, it would have been a good idea to take a smaller sample size and then make use of the chosen research instrument for gathering information from lesser number of respondents so that the process of drawing inferences would have become easier for the researcher. For future research, it is essential to test if the moderation effects are present subsequent to the control of the salient gender identity of the consumers which comprise the feelings and the attitudes related to the biological sex of the individuals. This is vital as variations can take place in gender identity as per the situation and they might be primed externally (Steele Ambady, 2006). Secondly, in light of the positive effects consistently on brand equity by the androgynous brands, examination of the repositioning strategies of brand gender would be of interest. It also has to be figured out if the respective attributes of brand gender independent or they co-vary which might result in effects that may be either complementary or suppressor for some of the traits. Conclusion It can be concluded from the study that the researcher could have made improvements in both the reliability as well as the validity of the research. It is very difficult to test the reliability of the current research due to the large number of participants that are involved making retesting it difficult. Apart from this internal consistency is also difficult to establish. As far as validity is concerned, content as well as construct validity was ensured by the researcher but for further accuracy of the inferences drawn it would have been better if the sample size was reduced and consisted of people from nations. References Campbell, D.T. Stanley, J.C., 1963. Experimental and Quasi-Experimental Designs for Research. Chicago : Rand McNally. Creswell, J.W., 2003. Research Design: Qualitative, Quantitative and Mixed Methods Approaches. 2nd ed. Thousand Oaks. USA.: SAGE. Creswell, J.W., 2005. Educational Research: Planning, Conducting and Evaluating Quantitative and Qualitative Research. 2nd ed. Pearson Merrill Prentice Hall. Dolnicar, S. Rossiter, J.R., 2008. The low stability of brand-attribute associations is partly due to market research methodology. International Journal of Research in Marketing, 25(2), pp.104-08. Joppe, M., 2000. The Research Process. [Online] Available at: https://www.ryerson.ca/~mjoppe/rp.htm [Accessed 7 September 2016]. Kirk, J. Miller, M.L., 1986. Reliability and validity in qualitative research. Sage Publications.: Beverly Hills. Lieven, T. Hildebrand, C., 2015. The impact of brand gender on brand equity Findings from a large-scale cross-cultural study in ten countries. International Marketing Review, 3(2), pp.172-95. Moskal, B., Leydens, J. Pavelich, M., 2002. Validity, reliability and the assessment of engineering education. Journal of Engineering Education , 91(3), pp.351-54. Steele, J.R. Ambady, N., 2006. Math is hard! The effect of gender priming on womens attitudes. Journal of Experimental Social Psychology, 42(4), pp.428-36.