AbstractPoint-of-care ultrasound (POCUS) is a valuable tool in the prehospital management of critically ill patients, particularly patients with trauma, dyspnea, or shock. This review aims to evaluate the diagnostic and therapeutic applications, limitations, and implementation challenges of POCUS in prehospital care. Key findings are that ultrasound, particularly the extended Focused Assessment with Sonography in Trauma (eFAST) protocol, offers high specificity in identifying severe hemorrhage in trauma cases, although its sensitivity varies depending on the clinical context and operator experience. In dyspnea, pulmonary ultrasound significantly enhances diagnostic accuracy, aiding early detection of heart failure and other respiratory conditions. For shock, focused echocardiography facilitates rapid diagnosis and timely therapeutic decisions, improving patient outcomes. However, the effectiveness of POCUS is highly dependent on the operator’s expertise, and challenges such as time, space, and resource limitations in prehospital settings may impact its use. Furthermore, local studies in Latin America assessing the impact of prehospital ultrasound on morbidity and mortality reduction are lacking. Based on our review, we recommend standardized training programs, increased availability of portable ultrasound devices, and prospective studies on cost-benefit analysis to optimize POCUS implementation in prehospital systems, particularly in resource-limited regions. Prehospital ultrasound has the potential to revolutionize patient care by improving diagnostic precision and reducing time to definitive treatment, but its successful implementation requires strategic integration of technology, education, and research.
INTRODUCTIONPoint-of-care ultrasound (POCUS) refers to performing ultrasound based on patient signs or symptoms at the patient’s bedside. It has become a crucial resource in patient care within emergency services and prehospital contexts [1]. Worldwide, many prehospital services have implemented the use of POCUS as a diagnostic and therapeutic support mechanism within their protocols, recognizing that timely care for critically ill patients can be beneficial [2,3].
Literature describing the use of prehospital ultrasound for specific diagnostic and therapeutic interventions is accumulating rapidly, though some studies have raised concerns about the extrapolation of in-hospital sensitivity, specificity, and accuracy of POCUS to prehospital use [4]. Ketelaars et al. [5] described the ABCDE (airway, breathing, circulation, disability, and exposure) approach to ultrasound in prehospital settings, highlighting its usefulness in the initial diagnosis and treatment of traumatized or critically ill patients. By analyzing each component of the protocol (airway, breathing, circulation, disability, and exposure), the authors detailed how ultrasound enables the identification of critical injuries, optimizes interventions, and improves the accuracy of early decision-making. Their review supports the use of portable ultrasound as a key tool for immediate diagnosis in emergency situations, through ultrasound techniques like extended Focused Assessment with Sonography in Trauma (eFAST) for evaluating internal bleeding and abdominal trauma, pulmonary ultrasound for detecting pneumothorax and other respiratory complications, and focused echocardiography to assess cardiac function and circulatory status is crucial in patients with shock, among others.
In Latin American countries, it is uncommon to find prehospital services equipped with POCUS systems. Reasons for this vary, from the cost of acquiring equipment to staff training, which, in fact, is highly variable across Latin America. Similarly, the lack of local studies showing reductions in morbidity and mortality with POCUS application in various prehospital scenarios contributes to the scarcity of POCUS in prehospital settings.
Our objective was to conduct a structured search of the literature using keywords and perform a narrative review of the literature to describe the following: (1) the environments where prehospital ultrasound is used; (2) clinical conditions for using prehospital ultrasound; and (3) current training and education strategies for prehospital staff. Our aim with this review was to complement other more specific reviews on prehospital ultrasound using the following guiding research question, “What are the current environments of use, clinical applications, and key considerations or barriers to the implementation of ultrasound by emergency medical service personnel?”
METHODSThis review was conducted using the scoping review methodology, following the guidelines established by the JBI protocol [6] and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist [7], designed to enhance transparency and quality in systematic reviews and meta-analyses (Supplementary Material 1).
Eligibility criteriaStudies that met all of the following criteria were included: (1) patients who received ultrasound in a prehospital setting; (2) full-text articles published in indexed journals; (3) original research, including randomized controlled trials, nonrandomized controlled trials, and cohort clinical trials; and (4) articles published in the last 5 years (2019–2024). The following were excluded: (1) duplicate articles, those without a visible abstract in Rayyan (Rayyan; https://www.rayyan.ai/), or those not downloadable for full evaluation; (2) abstracts or full-text articles published as brief communications, letters to the editor, guidelines, websites, or case reports; (3) studies conducted in animals or simulators; and (4) articles in languages other than English or Spanish.
Search strategyRelevant studies were identified through an electronic search in the PubMed, Scopus, and ScienceDirect databases. The keywords "ultrasonography," "prehospital," and "emergency medical services" were used (Table 1). The search was limited to articles published in English and Spanish during the last 5 years (2019–2024).Data extraction (selection and coding)The data extraction process was carried out by two investigators who collected the information; a third investigator verified whether the data met all criteria. Searches were then merged into the Rayyan web application [8], where duplicates were removed and studies were selected for title and abstract evaluation. An analysis of the selected articles was carried out to determine if they met the inclusion and exclusion criteria in a Microsoft Excel (Microsoft Corp) matrix by the investigator. This was done to identify studies that met the inclusion and exclusion criteria and remove duplicate records.
Risk of bias (quality) assessmentAfter data extraction, agreement and disagreement among the selected articles' content were reviewed by two of the investigators. Discussion and consultation with a third evaluator were held in cases of disagreement among the authors. This also helped avoid selection bias in the included articles/studies.
RESULTSThe literature review identified 8,615 studies from the three mentioned databases. Of these, 533 records were eliminated before the selection phase due to duplicates and the Rayyan automation tool. After limiting the search to articles published between 2019 and 2024, 8,082 records were evaluated by title and abstract, of which 8,020 were excluded (158 by automation tools and 7,862 by investigator selection) A total of 62 studies were read in full text, with 46 studies excluded for not meeting inclusion criteria or having one of the exclusion criteria, leaving 16 articles for comprehensive analysis (Fig. 1).
The included articles address scenarios and strategies for ultrasound use in prehospital care, grouped as shown in Table 2. Results are presented by these groupings. Additionally, bibliographic features are presented and organized in Table 3 [2,3,9–22].
Trauma and eFASTFive articles evaluated the performance, sensitivity, and specificity of eFAST as a diagnostic strategy for trauma patients in prehospital settings, with a total of 1,740 patients evaluated [2,3,9–11]. These studies retrospectively assessed records taken by prehospital personnel (doctors and paramedics) of trauma victims and compared them to images taken at the receiving emergency center (ultrasound, computed tomography [CT] scan, or surgical findings).
Gamberini et al. [9] reported performing prehospital eFAST in 44 of 199 patients, resulting in identification of 27 positive cases compared to 128 positive cases found in the hospital. They described a general sensitivity of 62.9% and specificity of 100% for positive findings. Additionally, they reported that patients with positive prehospital eFAST findings had a significantly shorter time to CT or surgery (46 minutes vs. 69 minutes, P<0.001). They also noted a significant relationship between eFAST and the Injury Severity Score (ISS) with this time reduction compared to clinical data like hypotension or the Glasgow Coma Scale.
Lucas et al. [10] assessed whether prehospital eFAST influences prehospital strategy and time to surgical treatment based on a randomized controlled trial involving 242 victims. The prehospital team performed a clinical exam or clinical exam supported by eFAST according to a schedule. The results showed that the eFAST-supported clinical exam had better sensitivity and specificity than clinical examination alone (sensitivity of 94.7% and specificity of 97%, compared to 80%). Furthermore, the mean time to hospital admission was significantly reduced by 13 minutes, and time to surgical treatment by 15 minutes with eFAST.
Oliver et al. [2] performed prehospital eFAST on 361 patients, with findings later compared to a CT scan. They reported a sensitivity of 28% and specificity of 98%. Similar data were reported by Stralec et al. [11], who evaluated eFAST performance in detecting severe hemorrhage in 527 patients with an ISS of 11. Fourteen percent of patients had severe hemorrhage, with a positive eFAST result in 38% (sensitivity, 38%; specificity, 87%).
Partyka et al. [3], in a multicenter study involving 411 patients, reported the accuracy of prehospital eFAST for detecting peritoneal fluid (sensitivity, 25%; specificity, 96%), pneumothorax (sensitivity, 38%; specificity, 96%), hemothorax (sensitivity, 17%; specificity, 97%), and pericardial effusion (sensitivity, 17%; specificity, 100%). Most patients were male (73%), were closed trauma cases (98%), and two-thirds were injured in traffic accidents.
DyspneaFour publications focused primarily on the diagnosis of respiratory (cardiac or pulmonary) alterations related to the presence of dyspnea as a presenting symptom using lung ultrasound (LUS) [12–15]. Experienced paramedics in LUS evaluated a total of 44 patients with symptoms of dyspnea, and their results were compared with images taken at the receiving hospital [12]. A match of 90.91% was found, with no statistical differences between prehospital care and the emergency department. The same study reports a mean time of 63.66 seconds to obtain the LUS.
A prospective interventional study on patients transported to a hospital by a prehospital system included patients diagnosed with dyspnea as their primary symptom. Sensitivity and specificity for acute heart failure by paramedics were 23.3% and 97.2%, respectively, when LUS was not used, and 71% and 96%, respectively, when LUS was used. The most common alternative final diagnoses were chronic obstructive pulmonary disease (32%), pneumonia (13%), and asthma (8%). Overall, sensitivity increased by 48% without affecting specificity [13].
Gundersen et al. [14] similarly evaluated a group of 214 patients presenting with dyspnea as a symptom during prehospital care. Forty prehospital physicians initially assessed the patients and formed a diagnostic suspicion, then performed LUS and revised their diagnosis. Final records were evaluated by experts who determined the quality and accuracy of the images and diagnosis. Acute heart failure was suspected in 64 of 214 patients (30%) before ultrasound and in 64 of 214 (30%) after ultrasound. Ultrasound led to reclassification in 53 of 214 patients (25%).
Pietersen et al. [15], in their study related to ultrasound in patients with respiratory symptoms, collected 406 ultrasound images taken by paramedics during prehospital care between November 2018 and April 2020 in Denmark. Prior to this, prehospital personnel were trained in thoracic ultrasonography to conduct a retrospective quality control assessment with an evaluator who scored the quality and appropriateness of each image. The result of the review was 87.0% normal, 89.9% for interstitial syndrome, 97.3% for possible pneumothorax, and 96.3% for pleural effusion, with adequate image quality to determine whether pulmonary pathology was present or not.
Cardiopulmonary resuscitationThe use of ultrasound during cardiopulmonary resuscitation (CPR) maneuvers has recently expanded with the advent of portable devices. For out-of-hospital CPR, the literature is still in a growth phase. We identified one study correlating expired CO2 with ventricular contraction measured by ultrasound [16]. This study evaluated 18 patients, and during resuscitation maneuvers, continuous end-tidal CO2 (ETCO2) was measured along with subxiphoid ultrasound at least two or three times throughout the process. The ultrasound allowed for the evaluation of compression indices of the right and left ventricles and the correlation of these indices with ETCO2. Thoracic compressions induced significant compressions in all the observed cardiac chambers, such as the left ventricle (20.6%±13.8%) and the right ventricle (34.5%±21.6%). A positive correlation between ETCO2 and compression of the evaluated cardiac chambers was identified.
Prehospital ultrasound with remote assistanceRemote-assisted ultrasound uses telecommunications technologies to transmit images from a remote location to an expert who interprets them in real-time and guides prehospital diagnosis. Remote expert assistance for ultrasound evaluations performed in the prehospital setting promises to improve diagnostic success rates. We identified three publications that reported on prehospital ultrasound with remote assistance in our literature search [17–19].
Hermann et al. [17] obtained prehospital ultrasound images from patients who met any of these criteria: trauma, acute dyspnea, or circulatory failure. Of 24 patients, 58% presented with circulatory failure, with trauma in one case only. Remote supervision was successfully carried out in 17 of 24 cases: three cases did not receive timely supervision, and one case was not supervised due to connectivity issues. Audio issues were experienced in another three cases; these were resolved through a phone call to the expert and were satisfactorily resolved.
Siu et al. [18] evaluated the impact of remote supervision on prehospital personnel performance in performing eFAST in simulated cases. They compared eight supervised cases with eight independently assessed cases, with supervision times of 5 minutes 16 seconds and 8 minutes 27 seconds, respectively. The percentage of correctly identified injuries in the independent versus remote-supervised eFAST was 65% versus 92.5%, with a P-value of 0.01. They reported that prehospital personnel experienced significantly higher mental, temporal, and effort demands, greater frustration, and lower subjective performance during the independent trials.
Similarly, Engelsen et al. [19] evaluated the feasibility of remote-assisted ultrasound during airlift transfer by helicopter. In this case, the prehospital personnel consisted of four anesthesiologists and one military doctor who performed eFAST with remote supervision during nine helicopter flights, each with a unique healthy volunteer. A remote expert radiologist guided the doctors in their examinations. The obtained images were evaluated according to quality criteria described in the document, with an average rating of 4.9, corresponding to good diagnostic quality. The average duration of the teleguided eFAST was 5 minutes 54 seconds. Additionally, they described issues with signal transmission depending on proximity to urban areas and when flying above 2,000 ft (600 m).
Implementation of ultrasound and image recording in prehospital systemsThe implementation of ultrasound in prehospital services requires the integration of an image archiving system that allows images to be audited for statistical and practical data, such as the accuracy of image acquisition by prehospital professionals. Aziz et al. [20] described the implementation of an image archiving system and a clinical governance framework in a UK helicopter emergency medical service. During the study period, 5,913 patients were treated, 18.6% of whom had registered POCUS exams. The image quality was similar between doctors and paramedics, with adequate concordance between reviewers and prehospital personnel.
Implementation of ultrasound in prehospital services presents economic, training, and maintenance challenges, among others. Alsulami et al. [21] explored various topics related to implementation, such as perceived barriers, transportation mode, ultrasound familiarity, and stakeholders' self-efficacy in implementing prehospital ultrasound in the Saudi Red Crescent. They found that the primary barrier was the cost of ultrasound equipment and maintenance; other barriers identified were lack of training, skills among staff, and environmental factors that interfered with image quality.
Impact on outcomesOne study evaluated patients who entered the emergency department who were treated by prehospital services and divided these patients into two groups: a group that received POCUS (n=169) and another group that received only standard care before arriving at the emergency department (n=671) [22]. Median length of stay in the emergency department for the prehospital POCUS group was 159 minutes, compared to 165 minutes in the standard care group (P=0.125). Patients evaluated with prehospital POCUS had a shorter stay than those who received only standard care, but without statistical significance.
DISCUSSIONPrehospital ultrasound is a valuable tool for managing critically ill patients, with notable applications in trauma, dyspnea, and shock. Our review found that ultrasound shows good specificity for identifying conditions such as pneumothorax or abdominal fluid, consistent with previous findings [22,23]. However, prehospital eFAST currently has insufficient sensitivity for predicting the presence of injuries compared to imaging performed within the hospital setting. This is likely because a positive eFAST result is highly indicative of a corresponding injury confirmed by CT scans. In critically ill patients, prehospital eFAST provides important information to tailor patient management and direct them to the most appropriate facility. However, negative prehospital eFAST results do not definitively rule out the presence of injuries, and additional studies should be performed once the patient is in the hospital to exclude potential injuries.
Akoglu et al. [23] reported a sensitivity of 40% to 75% for intrahospital eFAST compared to CT scans in identifying thoracic and abdominal injuries. This difference in sensitivity outcomes can be attributed to variation in the conditions experienced in prehospital settings, such as weather, lighting, and noise. Additionally, in the early minutes post-trauma, some conditions, such as hemoperitoneum, pneumothorax, or hemothorax, may still be in very early stages and too small to be detected by ultrasound during initial examination. By the time the patient reaches the hospital, these injuries may have progressed. A recent study recommended serial eFAST examinations as a follow-up method for suspected injuries in hospital settings [24].
Various authors have corroborated that ultrasound and its interpretation are operator-dependent [25,26]. Factors such as the operator's professional background, experience, and training can affect sensitivity and specificity results. While this has been documented in intrahospital research, more studies are needed to address this issue in prehospital environments. Additionally, it is well known that prehospital emergency systems are regulated heterogeneously across countries, with varying professional profiles, training levels, and skills.
Dyspnea can have multiple causes, including abnormalities in the respiratory, cardiac, muscular, or metabolic systems. A recent pilot study in the prehospital setting demonstrated that using ultrasound to evaluate B lines is effective in ruling out acute heart failure, although it lacks specificity to confirm it [14]. Conversely, another small-scale prehospital study indicated that detecting pleural effusion using ultrasound has high sensitivity and specificity for diagnosing acute heart failure [27]. Our review of the literature revealed significant improvement in diagnostic accuracy when ultrasound was used in patients with dyspnea, facilitating early identification of acute heart failure.
Future recommendationsFuture research on ultrasound in prehospital settings should include prospective, multicenter, and large-scale studies, especially in countries where publications on this topic are still limited. It is also essential to promote research that evaluates not only the clinical application of ultrasound but also its costs and benefits in prehospital scenarios and its integration with existing healthcare systems. Research efforts should be aligned with the development of training programs tailored to local needs. Standardized protocols will enable more efficient use of ultrasound, ensuring that prehospital professionals can enhance decision-making processes that positively impact the outcomes of critically ill patients.
ConclusionsPrehospital ultrasound is a promising tool with the potential to transform prehospital care by improving diagnostic accuracy and reducing time to definitive treatment. However, its implementation faces significant challenges, particularly in countries with limited resources. The future of prehospital ultrasound depends on comprehensive strategies that combine technology, education, and research to ensure that it has a positive impact on the care of critically ill patients.
NOTESAuthor contributions
Conceptualization: all authors; Data curation: EP; Formal analysis: JLPA, EP; Funding acquisition: EP, JLPA; Investigation: JLPA, EP; Methodology: all authors; Resources: JLPA, EP; Software: JDYP; Supervision, JLPA, EP; Validation, JLPA; Visualization: JLPA, EP; Writing–original draft: JLPA, EP; Writing–review & editing: all authors. All authors read and approved the final manuscript.
Supplementary materialsSupplementary materials are available from https://doi.org/10.15441/ceem.24.374.
Supplementary Material 1.PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist.
REFERENCES1. Nelson BP, Sanghvi A. Out of hospital point of care ultrasound: current use models and future directions. Eur J Trauma Emerg Surg 2016; 42:139-50.
2. Oliver P, Bannister P, Bootland D, Lyon RM. Diagnostic performance of prehospital ultrasound diagnosis for traumatic pneumothorax by a UK Helicopter Emergency Medical Service. Eur J Emerg Med 2020; 27:202-6.
3. Partyka C, Coggins A, Bliss J, et al. A multicenter evaluation of the accuracy of prehospital eFAST by a physician-staffed helicopter emergency medical service. Emerg Radiol 2022; 29:299-306.
4. Jorgensen H, Jensen CH, Dirks J. Does prehospital ultrasound improve treatment of the trauma patient? A systematic review. Eur J Emerg Med 2010; 17:249-53.
5. Ketelaars R, Reijnders G, van Geffen GJ, Scheffer GJ, Hoogerwerf N. ABCDE of prehospital ultrasonography: a narrative review. Crit Ultrasound J 2018; 10:17.
6. Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI manual for evidence synthesis. JBI; 2024.
7. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169:467-73.
8. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016; 5:210.
9. Gamberini L, Tartaglione M, Giugni A, et al. The role of prehospital ultrasound in reducing time to definitive care in abdominal trauma patients with moderate to severe liver and spleen injuries. Injury 2022; 53:1587-95.
10. Lucas B, Hempel D, Otto R, et al. Prehospital FAST reduces time to admission and operative treatment: a prospective, randomized, multicenter trial. Eur J Trauma Emerg Surg 2022; 48:2701-8.
11. Stralec G, Fontaine C, Arras S, et al. Is a positive prehospital FAST associated with severe bleeding?: a multicenter retrospective study. Prehosp Emerg Care 2024; 28:572-9.
12. Kowalczyk D, Turkowiak M, Piotrowski WJ, Rosiak O, Białas AJ. Ultrasound on the frontlines: empowering paramedics with lung ultrasound for dyspnea diagnosis in adults: a pilot study. Diagnostics (Basel) 2023; 13:3412.
13. Russell FM, Supples M, Tamhankar O, Liao M, Finnegan P. Prehospital lung ultrasound in acute heart failure: impact on diagnosis and treatment. Acad Emerg Med 2024; 31:42-8.
14. Gundersen EA, Juhl-Olsen P, Bach A, et al. PrehospitaL Ultrasound in Undifferentiated DyspnEa (PreLUDE): a prospective, clinical, observational study. Scand J Trauma Resusc Emerg Med 2023; 31:6.
15. Pietersen PI, Mikkelsen S, Lassen AT, et al. Quality of focused thoracic ultrasound performed by emergency medical technicians and paramedics in a prehospital setting: a feasibility study. Scand J Trauma Resusc Emerg Med 2021; 29:40.
16. Skulec R, Vojtisek P, Cerny V. Correlation between end-tidal carbon dioxide and the degree of compression of heart cavities measured by transthoracic echocardiography during cardiopulmonary resuscitation for out-of-hospital cardiac arrest. Crit Care 2019; 23:334.
17. Hermann M, Hafner C, Scharner V, et al. Remote real-time supervision of prehospital point-of-care ultrasound: a feasibility study. Scand J Trauma Resusc Emerg Med 2022; 30:23.
18. Siu M, Dan J, Cohen J, et al. Impact of telemedicine on extended focused assessment with sonography for trauma performance and workload by critical care transport personnel. Air Med J 2023; 42:105-9.
19. Engelsen PC, Heyerdahl F, Maddali DD, et al. Ascending with ultrasound: telementored eFAST in flight: a feasibility study. Emerg Radiol 2024; 31:25-31.
20. Aziz S, Edmunds CT, Barratt J. Implementation of a point-of-care ultrasound archiving system and governance framework in a UK physician-paramedic staffed helicopter emergency medical service. Scand J Trauma Resusc Emerg Med 2024; 32:49.
21. Alsulami M, Almukhlifi Y, Alsulami A, Al Nufaiei ZF, Alruwaili A, Alanazy A. Implementing prehospital ultrasound at the Saudi red crescent authority: perceived barriers and training needs. J Multidiscip Healthc 2024; 17:2871-8.
22. Ienghong K, Cheung LW, Tiamkao S, Bhudhisawasdi V, Apiratwarakul K. The impact of prehospital point of care ultrasounds on emergency patients length of stay in Thailand. J Multidiscip Healthc 2023; 16:219-26.
23. Akoglu H, Celik OF, Celik A, Ergelen R, Onur O, Denizbasi A. Diagnostic accuracy of the Extended Focused Abdominal Sonography for Trauma (E-FAST) performed by emergency physicians compared to CT. Am J Emerg Med 2018; 36:1014-7.
24. Yazici MM, Yavasi O, Celik A, Altuntas G, Altuntas M, Bilir O, Ersunan G. The role of repeated extended FAST in patients with stable blunt thoracoabdominal trauma. Ulus Travma Acil Cerrahi Derg 2023; 29:553-9.
25. Ullah N, Bacha R, Manzoor I, Gilani SA, Gilani SM, Haider Z. Reliability of focused assessment with sonography for trauma in the diagnosis of blunt torso trauma. J Diagn Med Sonogr 2022; 38:60-70.
Table 1.Search terms Table 2.Scenarios and strategies for ultrasound use in prehospital care
Table 3.Bibliography features
eFAST, extended Focused Assessment with Sonography in Trauma; FAST, Focused Assessment with Sonography in Trauma; CT, computed tomography; ED, emergency department; CI, confidence interval; OR, operating room; ISS, Injury Severity Score; CEX, clinical examination; AUC, area under the curve; HEMS, helicopter emergency medical service; LUS, lung ultrasound; BLUE, Bedside Lung Ultrasound in Emergency; AHF, acute heart failure; POCUS, point-of-care ultrasound; EMT, emergency medical technician; IQR, interquartile range; CPR, cardiopulmonary resuscitation; RV, right ventricle; LV, left ventricle; EtCO2, end-tidal carbon dioxide. |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||