Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.

Browsing by Author "Jonathan, Christopher"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Towards an Extensible Expert-Sourcing Platform
    (2019-05) Jonathan, Christopher
    In recent years, general purpose crowdsourcing platforms, e.g., Amazon Mechanical Turk, Figure Eight, and ChinaCrowds, have been gaining a lot of popularity due to their capability in solving tasks that are still difficult for machines or computers to solve, e.g., labeling data, sorting images, computing skyline over noisy data, and sentiment analysis. Unfortunately, current crowdsourcing platforms are lacking a very important feature that is desired by many of the recent crowdsourcing applications, namely, recruiting workers that are expert at a given task. Being able to recruit expert workers will allow those applications to not only achieve a more accurate results but also higher quality results than recruiting general crowd for the applications. We call such crowdsourcing process as expert-sourcing, i.e., outsourcing tasks to experts. Without having any platforms to support them, developers of each expert-sourcing application needs to build the whole crowdsourcing system stack from scratch while, in fact, those systems share many common components with each other. This thesis proposes Luna; the first extensible expert-sourcing platform. To instantiate a new expert-sourcing application out of Luna, one only needs to provide a few simple plug-ins that will be integrated with the core components of Luna to provide the expert-sourcing platform for the new application. This is possible due to the fact that Luna is able to identify the components that can be shared among many expert-sourcing applications and the components that need to be tailored for a specific application. In this thesis, we show the extensibility of Luna by instantiating six different expert-sourcing applications that are currently not well supported by the general purpose crowdsourcing platforms. Experimental evaluation with real crowdsourcing deployment as well as by using real dataset shows that Luna is able to achieve not only more accurate but also better quality results than existing general purpose crowdsourcing platforms in supporting expert-sourcing applications. Lastly, we also provide a more specialized expert-sourcing platform for image geotagging application that is initially deemed unfit to be solved by crowdsourcing.