Pattern-Based Specification of Crowdsourcing Applications – ICWE 2014 best paper

I’m really proud to announce that our paper “Pattern-Based Specification of Crowdsourcing Applications” has received the BEST PAPER award at ICWE 2014 (International Conference on Web Engineering), held in Toulouse in July 2014. The paper was authored by Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Andrea Mauri, and Riccardo Volonterio.

The work addresses the fact that in many crowd-based applications, the interaction with performers is decomposed in several tasks that, collectively, produce the desired results.
A number of emerging crowd-based applications cover very different scenarios, including opinion mining, multimedia data annotation, localised information gathering, marketing campaigns, expert response gathering, and so on.
In most of these scenarios, applications can be decomposed in tasks that collectively produce their results; Tasks interactions give rise to arbitrarily complex workflows.

In this paper we propose methods and tools for designing crowd-based workflows as interacting tasks.
We describe the modelling concepts that are useful in such framework, including typical workflow patterns, whose function is to decompose a cognitively complex task into simple interacting tasks so that the complex task is co-operatively solved.
We then discuss how workflows and patterns are managed by CrowdSearcher, a system for designing, deploying and monitoring applications on top of crowd-based systems, including social networks and crowdsourcing platforms. Tasks performed by humans consist of simple operations which apply to homogeneous objects; the complexity of aggregating and interpreting task results is embodied within the framework. We show our approach at work on a validation scenario and we report quantitative findings, which highlight the effect of workflow design on the final results.

Here are the slides presented by Alessandro Bozzon during the ICWE conference:


Here is Alessandro Bozzon presenting:

and here is the picture of the actual award:

ICWE 2014 Best Paper Award Certificate to Pattern-Based Specification of Crowdsourcing Applications. Bozzon, Brambilla, Ceri, Mauri, Volonterio

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

Featured on CrowdResearch blog (Follow the Crowd)

I’m happy to report that a post of ours on Reactive and Multiplatform Crowdsourcing has been featured on the renowned CrowdResearch blog (Follow the Crowd).
The editorial and advisory team of the blog is an exceptional one, including:

  • Michael Bernstein, Stanford University
  • Alex Quinn, University of Maryland
  • Ed H. Chi, Google Research
  • Rob Miller, MIT

and many others.

The post puts the focus on the need of controlling the crowd in response to  the quantity and timing of completed tasks;  the quality of responses and task results; and the profile, availability and reliability of performers.

To address this problem, we propose to integrate the following ingredients:
  • crowdsourcing, addressed with a modeling approach
  • social networking, as alternative means for outsourcing tasks on the web
  • multi-platform integration for allowing flexibility in task allocation
  • expertise finding for matching tasks to performers
  • reactive rules for declaring events and triggers that control the crowd.

The nice outcome is that by applying our methods you get significant advantage in terms of time, effort and quality. Some examples are reported in the figure below, showing up to 50% increase in precision of results and saving of more than 20% of executions, simply by changing the reactive rules for calculating majority.

Our post is somehow a summary of our three papers on crowd sourcing, as reported also in my previous post in this blog, namely:

All this is also summarized in the official CrowdSearcher web site:

The full post is available on the Crowdresearch blog.

 To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

Answering search queries with CrowdSearcher – WWW2012

Our paper:

 Answering search queries with CrowdSearcher
has been accepted and presented at WWW 2012 in Lyon.

Here is the abstract:
Web users are increasingly relying on social interaction to complete and validate the results of their search activities. While search systems are superior machines to get world-wide information, the opinions collected within friends and expert/local communities can ultimately determine our decisions: human curiosity and creativity is often capable of going much beyond the capabilities of search systems in scouting “interesting” results, or suggesting new, unexpected search directions. Such personalized interaction occurs in most times aside of the search systems and processes, possibly instrumented and mediated by a social network; when such interaction is completed and users resort to the use of search systems, they do it through new queries, loosely related to the previous search or to the social interaction. In this paper we propose CrowdSearcher, a novel search paradigm that embodies crowds as first-class sources for the information seeking process. CrowdSearcher aims at filling the gap between generalized search systems, which operate upon world-wide information – including facts and recommendations as crawled and indexed by computerized systems – with social systems, capable of interacting with real people, in real time, to capture their opinions, suggestions, emotions. The technical contribution of this paper is the discussion of a model and architecture for integrating computerized search with human interaction, by showing how search systems can drive and encapsulate social systems. In particular we show how social platforms, such as Facebook, LinkedIn and Twitter, can be used for crowdsourcing search-related tasks; we demonstrate our approach with several prototypes and we report on our experiment upon real user communities.

The full paper is available here:

The presentation I gave is this one:

The demo video can be found on YouTube:

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).