Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, appearance, and location makes the detection of polyps challenging. Moreover, colonoscopy surveillance and removal of polyps are highly operator-dependent procedures and occur in a highly complex organ topology. There exists a high missed detection rate and incomplete removal of colonic polyps. To assist in clinical procedures and reduce missed rates, automated methods for detecting and segmenting polyps using machine learning have been achieved in past years. However, the major drawback in most of these methods is their ability to generalise to out-of-sample unseen datasets from different centres, populations, modalities, and acquisition systems. To test this hypothesis rigorously, we, together with expert gastroenterologists, curated a multi-centre and multi-population dataset acquired from six different colonoscopy systems and challenged the computational expert teams to develop robust automated detection and segmentation methods in a crowd-sourcing Endoscopic computer vision challenge. This work put forward rigorous generalisability tests and assesses the usability of devised deep learning methods in dynamic and actual clinical colonoscopy procedures. We analyse the results of four top performing teams for the detection task and five top performing teams for the segmentation task. Our analyses demonstrate that the top-ranking teams concentrated mainly on accuracy over the real-time performance required for clinical applicability. We further dissect the devised methods and provide an experiment-based hypothesis that reveals the need for improved generalisability to tackle diversity present in multi-centre datasets and routine clinical procedures.
Widespread use of mobile devices has resulted in the creation of large amounts of data. An example of such data is the one obtained from the public (crowd) through open calls, known as crowdsourced data. More often than not, the collected data are later used for other purposes such as making predictions. Thus, it is important for crowdsourced data to be recent and accurate, and this means that frequent updating is necessary. One of the challenges in using crowdsourced data is the unpredictable incoming data rate. Therefore, manually updating the data at predetermined intervals is not practical. In this paper, the construction of an algorithm that automatically updates crowdsourced data based on the rate of incoming data is presented. The objective is to ensure that up-to-date and correct crowdsourced data are stored in the database at any point in time so that the information available is updated and accurate; hence, it is reliable. The algorithm was evaluated using a prototype development of a local price-watch information application, CrowdGrocr, in which the algorithm was embedded. The results showed that the algorithm was able to ensure up-to-date information with 94.9% accuracy.
Crowdsourcing introduces new perspectives in innovation, allowing for new products and services to shift away from the traditional manufacture-centric model to a more user-centric one. In order for businesses to reap the benefits of open innovation, it is necessary to understand the factors that motivate ideators to contribute valuable ideas. Equally, there is an urgency to identify the challenges faced by ideators in crowdsourcing for open innovation to retain the participants of crowdsourcing communities. This paper presents a structured review to address the aforementioned issues. Our findings reveal that the intrinsic factors that drive participation in open innovation are related to the learning experience that results from sharing ideas. Extrinsic factors like social motivation are frequently mentioned in different studies. This study also highlights the need for organisations to develop strategies for interacting with their contributors in order to sustain their participation and idea contribution. In conclusion, this paper can serve as a guideline for practitioners to improve crowdsourcing platforms with the inclusion of important motivational features. It can also serve as reference for organisations for formulating policies to regulate idea contribution.
Crowdsourcing has changed the way people conduct business. It provides access to work, and employers can source for the best talent, at the best price, with the shortest turnaround time. Research so far has focussed on crowdsourcing implementation. Hence, there is a need to conduct research that can contribute towards crowdsourcing sustainability. Thus, the objectives of this paper are to identify current practices of crowdsourcing in Malaysia and the challenges that face it. A conceptual model for crowdsourcing sustainability ecosystem is then proposed. This study adopted the case-study approach. Two crowdsourcing platforms were examined in the case study. Two techniques were used to obtain the data: observation and interview. Observation was carried out to observe how the crowdsourcing platforms worked. The interviews helped to uncover current practices, challenges in using crowdsourcing and identification of sustainability factors. It is hoped that the proposed conceptual model will facilitate better planning of the ecosystem supporting crowdsourcing and ensure sustainable growth for crowdsourcing. Future research into crowdsourcing can test the proposed conceptual model to validate its components.
Crowdsourcing is an initiative implemented by the Malaysian government to support its National Key Result Area (NKRA) agenda to improve the lives of citizens with low household income in the B40 group. Crowdsourcing activities are done on mobile crowdsourcing platforms that enable workers to perform micro tasks at any time for a fixed payment. However, without active and constant participation from the crowd, this initiative might not be successful. This paper describes a preliminary study in identifying motivation factors for participating in mobile crowdsourcing platforms. This study identified intrinsic and extrinsic motivation factors that can attract crowds to participate in mobile crowdsourcing platforms. Technology efficacy factors that interlink with motivation factors were also identified in this study. The preliminary study employed the qualitative method where in-depth interviews were conducted among 30 crowdsourcing participants in Peninsular Malaysia. The findings of this study are the basis for a motivation model that can attract crowdworkers from among the B40 group of household-income earners to participate in crowdsourcing to procure and perform available micro-tasks. The findings will also help improvise mobile platforms for crowdsourcing.
Test collection is used to evaluate the information retrieval systems in laboratory-based evaluation experimentation. In a classic setting, generating relevance judgments involves human assessors and is a costly and time consuming task. Researchers and practitioners are still being challenged in performing reliable and low-cost evaluation of retrieval systems. Crowdsourcing as a novel method of data acquisition is broadly used in many research fields. It has been proven that crowdsourcing is an inexpensive and quick solution as well as a reliable alternative for creating relevance judgments. One of the crowdsourcing applications in IR is to judge relevancy of query document pair. In order to have a successful crowdsourcing experiment, the relevance judgment tasks should be designed precisely to emphasize quality control. This paper is intended to explore different factors that have an influence on the accuracy of relevance judgments accomplished by workers and how to intensify the reliability of judgments in crowdsourcing experiment.
The Text Forum Threads (TFThs) contain a large amount of Initial-Posts Replies pairs (IPR pairs) which are related to information exchange and discussion amongst the forum users with similar interests. Generally, some user replies in the discussion thread are off-topic and irrelevant. Hence, the content is of different qualities. It is important to identify the quality of the IPR pairs in a discussion thread in order to extract relevant information and helpful replies because a higher frequency of irrelevant replies in the thread could take the discussion in a different direction and the genuine users would lose interest in this discussion thread. In this study, the authors have presented an approach for identifying the high-quality user replies to the Initial-Post and use some quality dimensions features for their extraction. Moreover, crowdsourcing platforms were used for judging the quality of the replies and classified them into high-quality, low-quality or non-quality replies to the Initial-Posts. Then, the high-quality IPR pairs were extracted and identified based on their quality, and they were ranked using three classifiers i.e., Support Vector Machine, Naïve Bayes, and the Decision Trees according to their quality dimensions of relevancy, author activeness, timeliness, ease-of-understanding, politeness, and amount-of-data. In conclusion, the experimental results for the TFThs showed that the proposed approach could improve the extraction of the quality replies and identify the quality features that can be used for the Text Forum Thread Summarization.
The increasing adoption of social media is a viable means in crowdsourcing. It can facilitate the connectivity of collaboration between different organisations, people and society to produce innovative and cost-effective solutions to many problems. Social media have opened up unprecedented new possibilities of engaging the public in meaningful ways through crowdsourcing. However, the growing number of security and privacy issues in social media may weaken the efficacy of crowdsourcing. This study aims to provide a basic understanding of security and privacy issues in line with the growth of crowdsourcing using social media platforms. This study also illustrates how crowdsourcing and social media data can lead to security and privacy issues in different environments. Lastly, this study proposes future works that may serve as direction for scholars to explore security and privacy in crowdsourcing through social media platforms. Secondary sources obtained from journals, conference papers, industry reports and books were reviewed to gather information.
Value Co-Creation (VCC) plays a major role in engaging knowledgeable individuals in a community via innovation, problem solving, and new service/product development. This study investigates the personal factors that influence individuals' engagement in value co-creation in Higher Education Institutions (HEIs) through the use of online platforms. Some higher education institutions have successfully established or used appropriate online platforms, such as online forums, web applications, and mobile applications to engage their community in ideation or crowdsourcing as a part of the value co-creation process. On the other hand, some HEIs have failed to engage their community in value co-creation activities, and even if they managed to engage some individuals in value co-creation once, they failed to sustain these individuals' engagement in value co-creation using online platforms. Using the Stimulus Organism Response (S-O-R) framework, this study examines the relationship between relevant personal factors (commitment and knowledge self-efficacy) and other motivational factors that provide perceived benefits with value co-creation engagement. Data was collected from 308 respondents at five Malaysian research universities. The software analysis tool Smart PLS is used for data analysis and validation. The results demonstrate that personal factors and perceived benefits as a motivational factor has a significant effect on individual engagement in value co-creation. However, the significance of these findings varies from one individual to another. The implications of these findings are discussed.