INTRODUCTION.

Colorectal cancer affects the large intestine (the colon), or the rectum, located at the distal end of the colon. In 2008, colorectal cancer yielded 610,000 deaths among high income countries [1]. In the United States, it is currently the second-leading cause of cancer-related deaths [2]. The presence of small benign growths in the colon, or polyps, suggest a potential for the development of colon cancer. The typical progression of cancer cells to malignancy is 5-10 years; however, in the special case of colon cancer, the removal of a polyp in the early, asymptomatic stage increases the chance of survival to 90% [3]. On the other hand, after the polyp has matured into malignancy, the survival rate dwindles to 5% [3]. With these drastic statistical differences in mind, the early detection of polyps is essential. Colonoscopy is currently viewed as the most effective and efficient method of screening, due to clear visualization of the lumen, possibility for biopsy acquisition, and lesion treatment capabilities [4, 5].

Though it is seen as a viable screening method, gastroenterologists are always searching for ways to improve colonoscopy procedures, especially the detection of polyps, to increase patient compliance. One improvement measure being taken is the utilization of teamwork within the colonoscopy procedures. Multidisciplinary teams, or MDTs, are becoming more popular in oncology in order to ensure that all patients receive the optimal diagnosis and therapy [6]. These MDTs utilize the expertise of several different professions as they attempt to give the best care possible. Previous studies concerning esophageal cancer suggest that MDTs are very effective. In one study, survival rates without the use of such a team were less than 20%, while the use of MDTs yielded survival rates topping 50% [7]. Teamwork in the operating room has taken on a similar appearance in colonoscopy procedures. Some gastroenterologists call on present professionals, such as nurses and other gastroenterologists, to give their input in the search for polyps, with the ultimate goal being to accurately identify as many polyps as possible. This teamwork also connects to the training process of medical students as prospective physicians begin to work with gastroenterologists in the search for polyps. A patient’s survival is dependent on the accurate identification of potentially harmful growths, which makes this initial teamwork, including that by way of student training, pertinent.

Though this teamwork attempts to improve the accuracy of polyp detection, there are still roadblocks. The primary setback is the setup of procedures in the operating room. Unfortunately, the endoscopic view is often streamed to one monitor, chiefly for the use of the acting gastroenterologist. This makes it difficult for other professionals to pinpoint potential polyps or regions of interest. Oftentimes, a polyp region can only be verbally described, rather than visually highlighted. This lack in technology hinders the process of medical teamwork and can prevent the gastroenterologist from effectively carrying out colonoscopy procedures, since the gastroenterologist must decipher what location is being described. Similar problems arise when gastroenterologists become teachers for medical students. As trainees are performing colonoscopies, it is often hard and time-consuming for the experienced gastroenterologist to visually pinpoint and verbally describe the missed polyps in an effective manner. This makes for a very cumbersome and disruptive process for all parties involved in the colonoscopy. This causes a great need for an innovative tool or resource that will improve the effectiveness of medical teams and enhance the ability of gastroenterologists to be teachers.

The primary purpose of this project is to resolve this issue by creating a tablet application that facilitates the medical teamwork in the operating room for more comprehensive polyp detection. Additionally, this research opens up the potential for a new educational tool that fosters a better learning environment for medical students. Beyond the primary purposes, this research also explores a method for wireless endoscopic video transfer from the endoscope to the tablet. As colonoscopies are rapidly being improved and innovated, this research not only works to aid in patient care and promote cancer prevention, but also adds to the knowledge pool of technology integration within colonoscopy procedures.

MATERIALS AND METHODS.

Application Development.

The tablet application based on Android software (Google, Mountain View, GA) was created in Eclipse (Ottawa, Ontario, CA) using Java coding language. Video frame processing features were developed with the use of the image processing library, OpenCV (Itseez, Nizhny, Novgord, Russia)[8]. The Android emulator encompassed in Eclipse was used for initial testing of the application. Later, the ASUS Transformer Tablet (ASUS, Beitou District, Taipei, Taiwan) was used to test the application on a physical tablet.

The application was developed to possess features for viewing the endoscopic stream and highlighting regions of interest where potential polyps may be present. The main activity of the application has a stream of the endoscopic view with a drawing view overlay on top it, allowing the user to freely draw or mark on the stream. The main activity also has five video stream control buttons and features such as patient and date identification fields for efficient use in the operating room.

Endoscopic Stream Transfer.

Endoscopic stream transfer includes frame-by-frame dispatching from the endoscope to the tablet. The architecture of the transfer builds on an already implemented endoscopic stream transfer protocol from lab personnel in other research projects [9], as well as implementing methods to transfer the video wirelessly from a computer to the tablet (Figure 1). From the endoscope, the endoscopic video streams to the image hub via S-Video, where each frame is extracted by the frame grabber and sent via USB to a Lenovo Think Pad (Lenovo, Morrisville, NC, USA). From the computer, each frame is then continuously sent to the tablet wirelessly, thus recreating the video. For this part of the research, a webcam was used to simulate the beginning stages of the endoscopic stream transfer. The webcam simulates the steps from the endoscope to the frame grabber and is directly connected via USB to the computer.

Figure 1. Image transfer setup. This project focuses on the steps in red.

An ad hoc network allowed for wireless communication between the computer and tablet. Transmission Control Protocol (TCP) was then implemented to create a socket connection between the computer and the tablet. In this TCP connection, the computer acted as the server, and the tablet act as the client. Eclipse is used to code the computer as a server and initiate the client status of the tablet within the application. During endoscopic streaming, the server is listening on the network on port 8888. After the user presses the button to start the endoscopic stream, the client connects to the socket on that port and the frame transfer is initiated. OpenCV library methods extract each frame from the endoscopic capture on the computer server and then send the image packet into socket output stream. The client side of the socket receives each image packet on the socket input stream and then displays the frame in the streaming view of the application. The user sees the recreated video that is comprised of the array of frames until the user presses the end connection button. Once this button is pressed, the endoscopic stream is ended upon socket closure.

Architecture Testing.

Following the design of the network transfer of the endoscopic stream, a modified version of the actual video transfer architecture shown in Figure 1 was tested. A laboratory-developed device with endoscopic features was connected to a Medigus IntroSpicio 3.0 mm camera system (Medigus LTD, Omer, Israel). The camera system was then connected to an Ezcap USB 2.0 Video Capture frame grabber (Ezcap.TV, Newton Stewart, United Kingdom) via S-Video. The frame grabber used USB port of the computer to send each frame to the server, and the aforementioned coding streamed the video from the server to the client.

RESULTS.

Application Development.

A tablet application for use in the operating room was successfully developed. The main activity encompasses all of the intended features (Figure 2). When a user enters into the main activity of the application, he has the option of pressing the START, ERASE, or END buttons. When the user presses the START button, the endoscopic stream appears on the screen as it receives the frames consistently from the server. This continues until the user presses the END button to discontinue the endoscopic stream. This action must be performed before exiting the application. In order to reconnect to the stream, the user must manually restart the server and reinitiate the socket connection. The ERASE button allows the user to utilize the actions of highlighting and marking the endoscopic stream. Once a potential polyp area is detected, the user can physically draw a circle or other marking symbol around the area. If the user is not qualified to discern an area as possessing a polyp with certainty, confirmation of the area can be received from the acting gastroenterologist. After the area has been verified, the user can press the ERASE button and reset the drawing view for the highlighting of another region of interest. During the procedure, the user may enter the person’s name in the respective field to later save images via the tablet’s screenshot capability. Also, during training, the gastroenterologist can track the student’s progress by pressing the +1 and -1 buttons under the detected polyps and missed polyps fields.

 

Figure 2. The layout of the main activity of the application. This represents the skeleton of the mechanism that will be used to mark a region of interest for a potential polyp.

Endoscopic Stream Transfer and Architecture Testing.

The endoscopic stream transfer setup was successfully developed and connected over the ad hoc network. From the computer, the frames of the webcam video were successfully sent to the tablet consistently. However, time of arrival did pose a problem, as the endoscopic stream on the tablet lagged behind real time movement. The socket connection was successfully closed upon the pressing of the END button in the main activity.

The actual architecture implementation also worked effectively. The endoscopic video stream successfully transferred from the lab-created version of an endoscope through the image hub, frame grabber and computer to finally appear on the tablet. However, issues with real-time transfer also arose with the use of the operating room equipment. Real-time characteristics were retained from the endoscope to the computer, but the frames began to lag upon their arrival to the tablet at only 1.7 frames/second.

DISCUSSION.

Mobile devices such as tablets have been increasingly integrated into the medical field [10]. However, the functions have mostly been limited to keeping patient records, management guidelines, and other data accumulation implementations [10]. The use of mobile devices as diagnostic tools has not always been viewed as feasible by professionals [10], but as technology improves and as this research shows, the potential for a broadening of tablet use in medicine is promising. The work of this research is a testament to the improving technology and application that allows for an improvement in technological integration.

The findings in this study reveal the feasibility of transmitting video from the endoscope to a tablet with the developed Android application. Specifically, this work shows the adaptability of live streaming video to a drawing view overlay for marking and highlighting within an Android environment. On a transmission level, the results of this work emphasize another application of the use of TCP socket connections to wirelessly connect a server to an external device, acting as a client, and relay frame data continuously. Most prominently, this research presents modern technology that is adapted to the needs of the medical field, and with further research, could be effectively integrated into the field’s existing procedures. This technology is not limited to the colonoscopy field as it presents a versatile characteristic that allows it to be integrated into a wide array of medical fields that seek for learning tools and collaboration in surgery.

Though the results do support the viability of using tablet technology for viewing and manipulating endoscopic video, improvements to the current technology can be made to enhance its quality. In subsequent versions, this technology will include the development of pausing and resuming capabilities to enhance efficiency of use. While it is not essential to the current goal of this application, it will allow the user to pause the reading of frames when a region of interest is pinpointed, and make remaining in a fixed position while tablet users examine a specific area of the colon unnecessary.

The primary target for improvement is increasing the frame rate of video stream, which thus allows it to possess full real-time characteristics. The problem has been pinpointed at the connection between the server and the client. Real-time characteristics are retained from the endoscope to the computer, but the transmission from the server to the client yields a reduction in frame receiving rate. This limitation could hinder the efficiency in the operating room, causing more disruption and a possible increased polyp miss rate. As such, efforts are currently being made to improve the frame receiving rate between the server and client to the classification of real-time, including sending each individual frame as a byte array and User Datagram Protocol integration. Additionally, one of the most promising efforts includes integration of a media player within the server and client that handles the transmission of video via the ad hoc network, rather than frame-by-frame transmission via sockets.

CONCLUSION/FUTURE WORK.

The results of the research supported the purpose of the project in developing a tablet application for comprehensive teamwork during colonoscopy procedures that implements endoscopic stream transmission. Furthermore, the development of the marking and highlighting feature is promising for resolving issues regarding the inability to visually highlight regions of interest on the endoscopic view.

Once the current technology quality has been further enhanced, clinical trials will be completed to assess the feasibility of tablet integration within an actual medical setting. Trials will be based on the previously-discussed notion that medical teamwork increases the survival rates and betters the treatment of cancer patients [6, 7]. The trials will be used to see if the developed tablet technology can enhance this medical teamwork specifically when dealing with medical students and gastroenterologist trainees. Munroe et al. used tandem colonoscopies in which trainees completed colonoscopy procedures, followed by experienced physicians repeating the colonoscopy procedures. Trainees in this study yielded a 27% miss rate of polyps, and only a 54% overall detection rate of polyps [11]. The implementation of the developed tablet will eliminate the need for tandem colonoscopies, as the gastroenterologist could view the endoscopic stream simultaneously on the tablet. With this added ability, clinical trials will be used to determine if the tablet presents a better learning method for trainees that would help them attain higher detection rates and lower miss rates, thus enhancing their study of the colonoscopy procedures. In clinical trials, the gastroenterologist will be following along on the tablet as the trainee is performing the colonoscopy, highlighting areas where a polyp is present. The tablet then becomes a tool for showing the student polyps that were potentially missed. In order to effectively carry out these clinical trials, further developments in screen sharing between the tablet and the monitor may be useful and necessary.

Mahmood and Darzi also completed studies related to the importance of feedback for trainees [12]. The research showed that no improvement was yielded for trainees using colonoscopy simulators when there was no feedback given, even after the amount of experience in the use of the increased [12]. Clinical trials will also be able to show whether the developed tablet will foster the environment that is conducive to the feedback that is pertinent to the success of trainees.

Tablet technology integration into colonoscopy explores a new arena of using technology in the medical setting. This work reveals a method for the development of tablet application for the purpose of streaming endoscopic view, thereby enhancing the quality and results of current colonoscopy procedures for the patient, gastroenterologist, and trainee.

ACKNOWLEDGMENTS.

Vanderbilt STORM Lab: Charreau Bell, Dr. Keith Obstein, and Dr. Pietro Valdastri. SSMV: Mr. Gabriel Sterling and Dr. Mary Loveless.

REFERENCES.