Determinants of low socio-economic status and risk of Plasmodium vivax malaria infection in Panama (2009–2012): a case–control study
© Obaldia; licensee Biomed Central. 2015
Received: 16 July 2014
Accepted: 21 December 2014
Published: 21 January 2015
Identification of risk factors is important for the establishment of malaria elimination programmes tailored to specific regions. Type of house construction had been associated with increasing risk of acquiring malaria. This study aimed at establishing the association between determinants of low socio-economic status (SES) and type of house construction with the likelihood of living in a Plasmodium vivax malarious corregimiento (smallest political division) in Panama during 2009–2012.
To determine the association between type-2 houses (build with deciduous materials) and other determinants of low SES, with living in a malarious corregimiento, this study analyzed demographic and housing census data (2010), and malaria incidence aggregated at the corregimiento level (2009–2012), using a Spearman’s non-parametric correlation test to explore for associations, followed by a case–control study and a reduced multivariate logistic regression approach for confirmation.
A descriptive temporal and spatial analysis indicated that P. vivax in Panama was associated with Amerindian reservations. Moreover, this study demonstrated that a strong correlation (deleterious effect) existed between living in a malarious corregimiento and being exposed to a type-2 house (OR = > 1.0) (p < 0.001), while, it showed an inverse correlation for exposure to type-1 houses (protective effect) (build with permanent materials) (OR = < 1.0) (p < 0.001). In the same way, a significant association between exposure to type-2 houses and the outcome of living in a malarious corregimiento was found using a case–control study approach (Chi2 test = p < 0.001), that was confirmed applying a reduced multivariate logistic regression fitted model.
This study demonstrated that living in a P. vivax malarious corregimiento in Panama during 2009–2012 was strongly correlated with those corregimientos having a high proportion of type-2 houses. A multivariate logistic regression approach at the house and corregimiento level indicated a strong association of type-2 houses, dirt floors and illiteracy with the likelihood of living in a malarious corregimiento. It is expected that these findings will help implement a multi-sectorial approach for the elimination of malaria in poor areas of Panama where malaria is endemic, which emphasizes house improvements such as mosquito-proofing and socio-economic development.
KeywordsPlasmodium vivax Panama Meso-America Malaria Epidemiology House type Elimination Eradication
The recently launched global effort to eradicate malaria has been stimulated by a significant decrease of cases in sub-Saharan Africa [1-8]. With the advent of artemisinin-based combination therapy (ACT), the epidemiological situation of the disease has dramatically changed in some parts of the world; for instance, 10 out of 99 countries where malaria transmission exists have been brought to the brink of elimination . Nonetheless, the burden of malaria remains high among the least developed and poorest areas, and within disadvantaged populations . As malaria transmission declines and political commitment and allocation of resources dissipate, identification of risk factors and “hot spots” to target interventions will be increasingly need it.
While Plasmodium falciparum is the deadliest of the human malaria parasites; accounting for over 600,000 deaths worldwide each year, mainly in children less than five years old from sub-Saharan Africa , Plasmodium vivax is a major reason for morbidity within and outside of Africa [8,11]. In the Americas, P. vivax malaria remains a major problem in poor underdeveloped areas, predominantly in the Amazonian basin, while in other areas of low transmission, such as Meso-America, it is focalized in rural areas and mostly under control in urban settings . For example, in Panama where malaria transmission have been reduce to pre-elimination levels in the last decade , still 90% of malaria cases, mostly all due to P. vivax remain focalized to Amerindian reservations — population that only accounts for less than 10% of the total, but where 89.8% of the individuals were living under the poverty line, with less than $ 3.25 US dollars per day by 2012 .
Malaria has been correlated with low socio-economic development and poverty , and recent evidence suggests, that a causal pathway between poverty and malaria runs in both directions . Furthermore, it has been pointed out that low socio-economic status (SES) doubles the risk of clinical malaria, reducing the capacity of countries to enter the elimination phase and that type of house construction is associated with increasing risk of malaria infection [16-20]. It has also been shown that interventions that address household improvements, such as mosquito-proofing, are associated with a decrease in malaria incidence [16,21-23]. In Latin America, however, only a few studies on the association between type of house building materials and P. vivax malaria incidence have been carried out .
This study examines the independent association between selected determinants of low SES status including type of house, dirt floors, lack of potable water, lack of electricity, lack of sanitary facilities, unemployment and illiteracy, all determinants of poverty, with the risk of living in a P. vivax malarious corregimiento in a low transmission setting; with the hypothesis that in Panama type-2 houses (build with deciduous construction materials), were associated with living in a malarious corregimiento. Results of this study showed a strong statistically significant association between type-2 houses and living in a P. vivax malarious corregimiento during 2003–2012. It is expected that this study will help generate new hypothesis and implement public health interventions that addresses risk factors based on a multi-sectorial approach , emphasizing house improvements such as mosquito-proofing and socio-economic development of malaria endemic regions.
This study was carried out in the Republic of Panama. Located in the southern tip of Central America, Panama limits to the east with the Republic of Colombia, the west with Costa Rica, to the north with the Caribbean Sea and to the South with the Pacific Ocean. The country is divided into 10 provinces that comprise 631 corregimientos (smallest political division analog to a county in the USA), with four Amerindian reservations (Ngobe-Bugle, Kuna-Yala, Madugandi-Wargandi and Embera-Wounan); the latter that accounts for only 10% of the ~ 3.4 million inhabitants (2010 Census). Approximately 2/3 of the population of Panama lives in urban or sub-urban areas. The Ministry of Health is divided into 14 health regions that sometimes overlaps a province including: Panama East (EST), Panama West (OES), Panama Metro (MTR) and San Miguelito (MGT); Bocas del Toro (BTO), Chiriqui (CHI), Veraguas (VER), Ngobe-Bugle (NGO), Herrera (HER), Los Santos (LSA), Cocle (CLE), Colon (COL), Darien (DAR) and Kuna-Yala (KUN) (Ministry of Health of Panama, 2010).
Type of study
Summary of demographic and housing census data (2010 Census, Panama)
In order to determine the correlation between independent variables of low SES and type of house, with the risk of P. vivax infection by year (incidence) at the corregimiento level, a single Spearman’s non-parametric pair-wise correlation of the study variables was carried out using STATA10® (StataCorp LP, College Station, Texas, USA) or SAS-JMP® (SAS Institute Inc. Cary, NC, USA). Likewise, in order to visualize the trend of the correlations, scattered plots of yearly malaria incidence against proportions of type-1 and type-2 houses by corregimiento were generated using PRISM® (GraphPad Software, Inc., La Jolla, CA, USA) plotting software. Similarly, a case–control study was carried out to determine the association between exposure to type-1-5 households and other determinants of low SES with the outcome of living in a malarious corregimiento. Furthermore, in order to assess the impact of a public health intervention, by removing the exposure of interest on the likelihood of living in a malarious corregimiento, the attributable fraction for the exposed (AttFexp) and the attributable fraction for the population (AttFpop) were calculated for 2009–2012. Finally, to evaluate the association between predictors of low SES and type-2 houses with the likelihood of living in a malarious corregimiento, a multivariate logistic model procedure was applied to estimate the odds of exposure to type-2 houses (binary: yes or no) compared to the odds of not being exposed, on the likelihood of living in a malarious corregimiento (binary: yes or no), controlling for confounders, such as dirt floors (continuous: proportion), lack of potable water (continuous: proportion), lack of sanitary facilities (continuous: proportion), unemployment (continuous: proportion) and illiteracy (continuous: proportion). A malarious corregimiento was defined as a corregimiento that presented one or more cases of P. vivax during the study period (2009–2012).
The logistic regression full model was defined as follows: logit (π i ) = ß o + ß 1Type1 house + ß 2Type-2 house + ß 3 X i + …ß x X i . Where (π i ) is the predicted probability for the ith corregimiento, of being infected with P. vivax. The odds of exposure to a type-2 house compared to the odds of not being exposed to a type-2 house, on the likelihood of living in a malarious corregimiento. An odds ratio of 1 mean that the odds for the two groups are the same, indicating that there is no effect. The significance of the OR was assessed using 95% confidence intervals (CI). Using a backward model selection a reduced model including only significant variables at the 0.05 level such as type-2 house, dirt floors and illiteracy was adopted for the final analysis. All statistical analyses were done using STATA10® (StataCorp LP, College Station, Texas, USA). Data were downloaded or transcribed to Filemaker® or Windows® Excel for descriptive and quantitative analysis.
Plasmodium vivax malaria incidence and case maps by year (2009–2012) at the corregimiento level were generated using the ArcMap® 10.2 mapping software (Redlands, CA, USA); in a similar way, in order to assess migratory patterns of P. vivax cases, 50 selected P. vivax cases were used to generate a trace back map using GIS coordinates and connecting lines from the location of the health facility reporting the case to the center of the case self-reported place (corregimiento) of origin using the X to Y toolbar from ArcMap® 10.2.
Data was obtained from publicly available secondary sources and analyzed at the corregimiento level (smallest political division) making impossible to identify the cases, therefore a human subjects protocol was not required.
Malaria is focalized to health regions associated with Amerindian reservations
More than 90% of malaria cases including P. falciparum and P. vivax infections by health region reported between 2003–2011 were found to be associated with the Amerindian reservations of the Ngobe-Bugle (NGO), Kuna-Yala (KUN), Madugandi-Wargandi (EST) and Embera-Wounan (DAR) (Additional files 1 and 2), with female individuals, less than 15 years old, accounting for 60.6% of the P. vivax cases compare to 38.3% of males in this age cohort during 2009–2012 (Additional files 3 and 4).
Plasmodium vivax incidence is highest among corregimientos associated with Amerindian reservations
Determinants of low SES and type-2 houses are associated with increase risk of P. vivax infection
Case–control study odds and 95% confidence intervals on the likelihood of living in a malarious corregimiento given exposure to determinants of low SES and type of house in Panama during 2009-2012
Odds ratio (95% confidence interval)
w/o potable water
Multivariate logistic regression model odds of exposure to selected determinants of socio-economic status and type of house on the likelihood of living in a malarious corregimiento in Panama during 2009-2012
Without potable water
This study demonstrates the association between selected determinants of low socio-economic status such as type-2 houses (build with temporary or deciduous construction materials), dirt floors and illiteracy with the likelihood of living in a P. vivax malarious corregimiento (smallest political division) during 2009–2012 in Panama.
In the same way, the study suggests that Panama is in a pre-elimination stage and that malaria is focalized to health regions (“hot spots”) associated with Amerindian reservations; where ~ 90% of malaria cases including both P. falciparum and P. vivax infections occurred during 2003–2012. This observation is supported by a focal low incidence of malaria by corregimiento (<1 – 8.3%) observed during 2009–2012. For instance, by 2011 the disease was under control with less than 500 cases per year, mostly due to P. vivax and localized to regions associated with Amerindian reservations (Figure 1).
However, as economic development reaches these focal areas of active malaria transmission, people tend to migrate looking for better jobs or education, bringing along parasites that seed new transmission foci to areas where malaria had been previously eliminated. Interestingly, part of the population of these reservations are dedicated to subsistence agriculture, but others are seasonal migrant agricultural workers dedicated to labour in the coffee, bananas or sugar cane plantations —Populations that migrate westward to the highlands of the province of Chiriquí for the coffee harvest season, or to the central provinces of Coclé, Veraguas and Herrera to harvest sugar cane. Using GIS mapping software 50 P. vivax cases were trace back to their places of origin, detecting movement of cases from as far as the eastern Provinces of Panama and Darien, but also, the western provinces of Bocas del Toro and Chiriquí bordering Costa Rica, towards the central provinces and to Panama City where malaria has been eliminated. These highly movable populations will have to be further investigated to account for other patterns of internal, continental and extra-continental illegal immigration at the moment of designing malaria control and eradication strategies (Figure 2).
Intriguingly, when P. vivax cases were stratified by age and gender for the period 2009–2012, the study found that female individuals less than 15 years old presented the highest number of P. vivax cases (60.6%) compared to males (38.3%). This finding contrast with recent reports from a moderate transmission setting in Tanzania, where male individuals in this same age cohort presented the highest malaria seroprevalence . However, anecdotal reports indicate that girls in the Kuna reservation of Madugandi-Wargandi in lake Bayano, engage in leisure and domestic activities such as sewing, laundry and cooking near or around the household, while young males in this age cohort are dedicated mainly to agricultural, hunting or fishing activities away from home. Activities that are in part associated with the lake and rivers banks, where houses are prevalently located —both areas of known breeding sites for Anopheles albimanus mosquitos ; exposing them differentially to the vector . This finding suggests differential malaria transmission based on age, gender and cultural or behavioural risk factors that should be further investigated.
In areas where malaria is endemic, house design and building are based on local experience, customs and available materials . Type-2 houses —the most prevalent type of house in the Amerindian reservations of Panama—, known locally as “bohíos” or “ranchos”, are build based on the palm tree and local experience, usually over dirt floors or raised above ground in stilts, to avoid seasonal flooding or insect nuisance . This type of house don’t offer much protection against the elements or vectors of disease unless are raised above ground , or its occupants use a mosquito net, but allows air flow for comfort. For example, a recent survey at the household level in The Gambia showed that children from the poorest quintiles living in houses build with poor walls, roofs and windows construction materials, were associated with a high prevalence of malaria , though, in the general population only poor wall housing materials were associated with a high prevalence.
The fact that during a recent survey at the Madugandi reservation in lake Bayano, only 7% of households reported using mosquito nets [Caceres L, Rovira J, Torres R, Calzada JE, Victoria C, Griffith M: Conocimientos, actitudes y practicas sobre la malaria en la poblacion indigena guna de la Comarca de Madugandi, Panama. Submitted, 2014], might help explain why in this study type-2 houses being the most prevalent in the Amerindian reservations of Panama, were highly correlated and associated to P. vivax malarious corregimientos. Suggesting that transmission occurs inside or closely associated with the house. Albeit, A. albimanus is predominantly exophagic and exophilic, there is evidence in the region of indoor resting after feeding . In the same way, the strong association between illiteracy and P. vivax infection might be explain by a 30% illiteracy [Caceres L, Rovira J, Torres R, Calzada JE, Victoria C, Griffith M: Conocimientos, actitudes y practicas sobre la malaria en la poblacion indigena guna de la Comarca de Madugandi, Panama. Submitted, 2014] and 69.5% extreme poverty rates reported for the Amerindian reservations in recent surveys . In contrast, the indication that dirt floors were protective of living in a P. vivax malarious corregimiento remains unclear. For instance, only 6% of the houses at the Madugandi reservation have cement floors [Caceres L, Rovira J, Torres R, Calzada JE, Victoria C, Griffith M: Conocimientos, actitudes y practicas sobre la malaria en la poblacion indigena guna de la Comarca de Madugandi, Panama. Submitted, 2014], Though, other confounding factors cannot be ruled out .
A limitation of this study is that it uses aggregated data at the corregimiento level for the multivariate correlations and logistic regression models and even though several confounding variables were adjusted for and the universe was the total houses and population of Panama, there could still be residual confounding from variables not accounted for. Moreover, ecologic study designs are prompt to the “ecological fallacy” when aggregated data (Group level) findings are generalized to the individual level [31,32]. The results of this study should be interpret interpreted with caution, though are useful preliminary indicators to causal hypothesis despite its methodological limitations when presented as a population level study because [33,34]: First, they are inexpensive and take little time to implement due to the availability of secondary sources (i.e. census databases); second, they are fast to analyse when compare to more elaborate and expensive individual level study designs (i.e. surveys). Moreover, they are particularly relevant when evaluating social processes or interventions at the population level (new programs, policy or legislation) . However, if the outcome at the group level is classified as dichotomous (as in the case–control study presented here) and the inferences are at the group level the study is not an ecologic study .
In the Americas, malaria remains a major problem in regions of low SES. In order to achieve elimination, current strategies must have a multi-sectorial approach that addresses social and environmental determinants of malaria infection. The current focal low transmission setting of malaria in Panama presents a unique opportunity to study the association between determinants of low SES and malaria at the population level to help guide interventions, policy and legislation. This study suggests that Panama is in a malaria pre-elimination stage and that transmission is low, focalized and mainly restricted to the Amerindian reservations. Furthermore, that type of house building materials and design play an important role in the transmission of P. vivax malaria in these endemic “hot-spots”. Increase risk of malaria infection was associated with those corregimientos having a high exposure to type-2 houses. Exposure to type-2 houses strongly increases the likelihood of living in a malarious corregimiento compare to not being exposed to a type-2 house. Type-2 houses, dirt floors and illiteracy were associated with the likelihood of living in a malarious corregimiento. Removing these exposures will likely contribute to accelerate malaria elimination in Panama. These results suggest that house improvements such as mosquito proofing and economic development of these areas may play an important role in a multi-sectorial approach for the control and elimination of malaria in Panama.
The author wish to thank, at Gorgas Memorial Institute of Health Studies in Panama, Nestor Sosa, Director and Jose E. Calzada for logistic support; at SENACYT, Ruben Berrocal, Secretary; Jorge Motta at Sistema Nacional de Investigacion, SNI-SENACYT and at Harvard School of Public Health, Professors Dyann F. Wirth, Chair, Sarah Volkman, Matthias Marti and Manoj Duraisingh for their encouragement and support to undertake these studies.
This study was supported in part by a doctoral fellowship from SENACYT-IFHARU, Panama, Panama; the Sistema Nacional de Investigacion de Panama (SNI), Panama; and the Department of Immunology and Infectious Diseases, Harvard School of Public Health, Boston, MA. USA.
The opinions and assertions contained herein are the private ones of the author and are not to be construed as official or reflecting the views of HSPH or ICGES.
- Bassat Q, Alonso PL. Defying malaria: fathoming severe Plasmodium vivax disease. Nat Med. 2011;17:48–9.PubMedView ArticleGoogle Scholar
- The malERA CGoM. A research agenda for malaria eradication: modeling. PLoS Med. 2011;8:1–9.Google Scholar
- Editorial. Is malaria eradication possible? Lancet. 2007;370:1459.Google Scholar
- Alonso PL, Brown G, Arevalo-Herrera M, Binka F, Chitnis C, Collins F, et al. A research agenda to underpin malaria eradication. PLoS Med. 2011;8:e1000406.PubMed CentralPubMedView ArticleGoogle Scholar
- Greenwood BM, Fidock DA, Kyle DE, Kappe SH, Alonso PL, Collins FH, et al. Malaria: progress, perils, and prospects for eradication. J Clin Invest. 2008;118:1266–76.PubMed CentralPubMedView ArticleGoogle Scholar
- Mendis K, Rietveld A, Warsame M, Bosman A, Greenwood B, Wernsdorfer WH. From malaria control to eradication: the WHO perspective. Trop Med Int Health. 2009;14:802–9.PubMedView ArticleGoogle Scholar
- Roberts L, Enserink M. Malaria. Did they really say … eradication? Science. 2007;318:1544–5.PubMedView ArticleGoogle Scholar
- Breman JG, Brandling-Bennett AD. The challenge of malaria eradication in the twenty-first century: research linked to operations is the key. Vaccine. 2011;29 Suppl 4:D97–D103.PubMedView ArticleGoogle Scholar
- WHO. World Malaria Report. Geneva: World Health Organization; 2012.Google Scholar
- UNDP RBMP. Multisectorial Action Framework for Malaria. RBM/UNDP: Geneva, Switzerland; 2013.Google Scholar
- Mendis K, Sina BJ, Marchesini P, Carter R. The neglected burden of Plasmodium vivax malaria. Am J Trop Med Hyg. 2001;64:97–106.PubMedGoogle Scholar
- Arevalo-Herrera M, Quinones ML, Guerra C, Cespedes N, Giron S, Ahumada M, et al. Malaria in selected non-Amazonian countries of Latin America. Acta Trop. 2012;121:303–14.PubMed CentralPubMedView ArticleGoogle Scholar
- Obaldia N, 3rd, Baro NK, Calzada JE, Santamaria AM, Daniels R, Wong W, Chang HH, Hamilton EJ, Arevalo-Herrera M, Herrera S, Wirth DF, Hartl DL, Marti M, Volkman SK: Clonal outbreak of Plasmodium falciparum infection in Eastern Panama. J Infect Dis. 2014. First published online October 21, 2014 doi:10.1093/infdis/jiu575Google Scholar
- Dieguez J, Alvarado R: Indigencia y pobreza: encuesta de mercado de trabajo. Panama, Republica de Panama, Ministerio de Economia y Finanzas ed. pp. 17. 2012.Google Scholar
- Sonko ST, Jaiteh M, Jafali J, Jarju LB, D’Alessandro U, Camara A, et al. Does socio-economic status explain the differentials in malaria parasite prevalence? Evidence from The Gambia. Malar J. 2014;13:449.PubMed CentralPubMedView ArticleGoogle Scholar
- Lindsay S, Jawara M, Paine K, Pinder M, Walraven G, Emerson PM. Changes in hosue design reduce exposure to malaria. Trop Med Int Health. 2003;8:512–7.PubMedView ArticleGoogle Scholar
- Gamage-Mendis A, Carter R, Mendis C, De Zoysa A, Herath P, Mendis K. Clustering of malaria infections within an endemic populations: risk of malaria associated with type of housing construction. Am J Trop Med Hyg. 1991;45:77–85.PubMedGoogle Scholar
- Yamamoto S, Louis VR, Sie A, Sauerborn R. Household risk factors for clinical malaria in a semi-urban area of Burkina Faso: a case–control study. Trans R Soc Trop Med Hyg. 2010;104:61–5.PubMedView ArticleGoogle Scholar
- Liu JX, Bousema T, Zelman B, Gesase S, Hashim R, Maxwell C, et al. Is housing quality associated with malaria incidence among young children and mosquito vector numbers? Evidence from Korogwe, Tanzania. PLoS One. 2014;9:e87358.PubMed CentralPubMedView ArticleGoogle Scholar
- Konradsen F, Amerasinghe P, van der Hoek W, Amerasinghe F, Perera D, Piyaratne M. Strong association between house characteristics and malaria vectors in Sri Lanka. Am J Trop Med Hyg. 2003;68:177–81.PubMedGoogle Scholar
- Lindsay S, Emerson PM, Charlwood J. Reducing malaria by mosquito-proofing houses. Trends Parasitol. 2002;18:510–4.PubMedView ArticleGoogle Scholar
- Ye Y, Hoshen M, Louis V, Seraphin S, Traore I, Sauerborn R. Housing conditions and Plasmodium falciparum infection: protective effect of iron-sheet roofed houses. Malar J. 2006;5:8.PubMed CentralPubMedView ArticleGoogle Scholar
- Bradley J, Rehman AM, Schwabe C, Vargas D, Monti F, Ela C, et al. Reduced prevalence of malaria infection in children living in houses with window screening or closed eaves on Bioko Island, equatorial Guinea. PLoS One. 2013;8:e80626.PubMed CentralPubMedView ArticleGoogle Scholar
- Danis-Lozano R, Rodriguez MH, Betanzos-Reyes AF, Hernandez-Avila JE, Gonzalez-Ceron L, Mendez-Galvan JF, et al. Individual risk factors for Plasmodium vivax infection in the residual malaria transmission focus of Oaxaca, Mexico. Salud Publica Mex. 2007;49:199–209.PubMedView ArticleGoogle Scholar
- Mosha JF, Sturrock HJ, Brown JM, Hashim R, Kibiki G, Chandramohan D, et al. The independent effect of living in malaria hotspots on future malaria infection: an observational study from Misungwi. Tanzania Malar J. 2014;13:445.View ArticleGoogle Scholar
- Caceres L, Rovira J, Garcia A, Torres R. Determinacion de la resistencia a insecticidas organofosforados, carbamatos y piretroides en Anopheles albimanus (Diptera: Culicidae) de Panama. Biomedica. 2011;31:419–27.PubMedView ArticleGoogle Scholar
- Mullen G, Durden L. Medical and Veterinary Entomology. 2nd ed. Burlington, MA, USA: Academic Press; 2009.Google Scholar
- Knudsen J, von Seidlein L. Healthy Homes in Tropical Zones: Improving Rural Housing in Asia and Africa. 2014.Google Scholar
- Charlwood JD, Pinto J, Ferrara PR, Sousa CA, Ferreira C, Gil V, et al. Raised houses reduce mosquito bites. Malar J. 2003;2:45.PubMed CentralPubMedView ArticleGoogle Scholar
- Kleinbaum D, Kupper L, Nizam A, Muller K. Applied regression analysis and other multivariable methods. Beltmont, CA, USA: Thomson Higher Education; 2008.Google Scholar
- Schwartz S. The fallacy of the ecological fallacy: the potential misuse of a concept and the consequences. Am J Public Health. 1994;84:819–24.PubMed CentralPubMedView ArticleGoogle Scholar
- Diez-Roux AV. Bringing context back into epidemiology: variables and fallacies in multilevel analysis. Am J Public Health. 1998;88:216–22.PubMed CentralPubMedView ArticleGoogle Scholar
- Morgenstern H. Ecologic Studies. 3rd ed. Philadelphia, PA, USA: Lippincott, Williams & Wilkins; 2008.Google Scholar
- Dohoo I, Martin W, Stryhn H. Ecological and Group Level Studies. VER Inc, Charlottetown, Prince Edward Island, Canada: Manitoba, Canada; 2009.Google Scholar
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.