High-Speed Idea Filtering With the Bag of Lemons

As part of their involvement in CATALYST, partners participate to research papers on Collective Intelligence based on the experience they gathered. Mark Klein (MIT / University of Zürich, CATALYST’s consortium partner), in collaboration with Ana Cristina Bicharra Garcia, published, as part of the official programme of the 2015 Collective Intelligence Conference their research results on the improvement of the idea filtering processes.

Based on the previously explored concepts of multi-voting and incentive providing, they created a new approach combining the two techniques. The article summarizes their methodology from the general concept to the lesson-learned via the experiment design and the evaluation results. This new approach, called Bag of Lemons, proved to allow a faster and more accurate idea filtering compared to existing models, such as the Likert scale, by asking participants to vote for the less convincing ideas based to pre-established criteria. It seems indeed to be much easier to eliminate an idea if at least one criterion is not reached whereas it is not always as simple to identify most perspicacious ideas when you are not a complete expert.

The full article is available here

Mark Klein, Massachusetts Institute of Technology, University of Zürich

Ana Cristina Bicharra Garcia, Universidade Federale Fluminense

Collective Intelligence 2015 Conference

CATALYST Final Event

CATALYST Final Event: meet us at CAPS2015

CATALYST will organize its final event within CAPS2015, the second edition of the annual event on Collective Awareness Platform for Sustainability and Social Innovation (CAPS), on July 7-8, 2015 in Brussels. CAPS2015 theme is ‘Networked Social Responsibility’ and CATALYST partners will bring on stage their lessons-learned and experience in the implementation of collective intelligence tools within online communities. The three sessions organized by CATALYST will consist in a roundtable, a workshop and interactive demos. The sessions are introduced below as well as on CAPS2015 website.

  • Harnessing the Power of Collective Intelligence: Technologies and Communities

Taking the form of a round table, this session will explain how to efficiently manage an online community, make sense of scattered conversation and move from deliberation to action. The CATALYST project built an entire ecosystem of collective intelligence open source tools to improve the quality and density of online conversation and deliberation. Integrating the tools within existing communities sparked a reflection around how to best build technologies at the service of the end-users and make them evolve further with a continuous feedback loop.

During this session, CATALYST partners will present the lessons-learned with the support of representatives from the testing communities (including Ashoka, OuiShare, Loomio and EdgeRyders).

  • Which Future for Collective Intelligence Processes and Solutions?

The past decade has seen the emergence of hundreds of innovative and powerful collective intelligence (CI) technologies and platforms, ranging from wikis to prediction markets, from social media mining to open source software development, from web forums to crowdsourcing platforms, and from argumentation systems to serious games for protein folding.

These tools have the potential to transform almost all aspects of our society in profound and positive ways, allowing us to work together to solve complex problems more effectively, drawing together human communities, and empowering the previously disenfranchised. The most important barrier to continuing progress in this realm is a two-folded one: on one side very few people have a clear sense of which tools to use for which purposes, or of how they should be used; on the other, many of these tools are abstract solutions because they have been developed without implying the ultimate users, and consequently do not satisfy or interpret correctly their needs. In this workshop animated by CATALYST partner attendees will be invited to join the debate about the future of collective intelligence developments.

  • Interactive Demos

This demo session will be interactive as participants will be able to get a demonstration of the whole ecosystem of tools developed within CATALYST through the “user journey” defined within the project. It is a new inclusive way to discover how the use of the tools can be articulated to improve the quality and the density of online debates and deliberations.


The whole preview of CAPS2015 programme is available on the event website and also include a workshop about Networks for Good, animated by Lee-Sean Huang, Strategist & Designer at Purpose Europe, and member of CATALYST consortium. The event is free but pre-registration is compulsory.


Follow CAPS2015 on Twitter @CAPS2020eu (use hashtag #CAPS15eu).

Testing CATALYST tools: an interview with Simon Tegg from Loomio

We interviewed Simon Tegg to understand his implication as software developer in the testing of CATALYST tools following Loomio’s successful application to the open-call.


Simon is a software developer at Loomio and Enspiral Craftwork. “At Loomio, we use online discussion and decision-making to manage the Loomio Workers Cooperative internally and engage with our community – using Loomio to build Loomio. The Loomio community has 700 people who we ask for advice on feature development and strategic direction”, he said. He further explained that these hundreds of members of Loomio community “are from all over the world and active in thousands of online communities”.


We then asked Simon about the reasons that pushed Loomio to apply to CATALYST open-call. He detailed his answer into two parts firstly explaining that “Collective intelligence is the idea that the intelligence of a group has the potential to be greater than any one group member and this was one of the main inspirations behind Loomio. He pointed out that the Loomio team experiences this every week in their own use of Loomio. “Someone will bring up a point that we had never thought of or put into words what we were thinking but didn’t know how to say and we’ll weave these points into the decision”, he said to support his statement.


Simon also mentioned that he and his community are now working on Assembl and Litemap as both tools “have taken approaches more explicitly geared towards large-scale collective intelligence. Our main interest in the CATALYST open call is as a shared learning project. We may incorporate any learnings into Loomio’s future development”, he claimed.


He continued explaining Loomio’s expectations from the use of CATALYST collective intelligence tools as “[They] know from [their] own experience that providing syntheses and ‘temperature checks’ at various stages can help groups move towards shared understanding. Assembl provides features for synthesising debates in a structured way. Loomio hopes to quantitatively test the impact these acts of facilitation has on the quality and comprehensiveness of discussion. The team is also intrigued by the impact that visualising the structure of the debate and presenting it back to participants will have as in the case of Litemap.”


Discover more interviews of CATALYST open-call winners such as the videos of Noemi Salantiu (EdgeRyders) and Carlos Rossique (AutoConsulta Ciudadana) on CATALYST YouTube channel and do not forget to subscribe for all live updates!

Testing CATALYST tools: an interview with Luca Iandoli from the University of Naples

We interviewed Luca Iandoli, an associate professor of Engineering Management at the School of Engineering of the University of Naples Federico II, to understand his implication in the testing of CATALYST tools following the University of Naples successful application to the open-call. Luca Iandoli is also a visiting professor at the School of Systems and Enterprises of the Stevens Institute of Technologies (Hoboken, NJ, USA) and a former Fulbright Visiting Scholar at the MIT Centre for Collective Intelligence. “In the last years I have focused my research work on collective intelligence and online collaboration. The fundamental question that I believe underlies most of my research work is: How can we design improve socio-technical systems in a way that individual action can be aggregated so that the collective system behaves in a more intelligent way?”, he precised.


Luca also shared with us information about his community, and what CATALYST tools can bring to it. “I have a large class of 150 engineering management students attending an introductory course in Economics and Management. A class is an obvious place for social learning and community emergence. The group is based in Naples, Italy and students are aged 21 to 23”. Luca also told us that he planned to use CATALYST tools to support extra lecture discussions and debate among students and in particular to help them to refine their argumentation skills. He supported his previous statement that “in more traditional engineering courses there is less room for debate and discussion because students are exposed to well known quantitative models and “unquestionable” design approaches. In Social Sciences instead, and in Economics in particular, theories and models tend to be more controversial and less amenable to strong quantitative modelling while the complexity of social systems can be much higher than the one of engineering systems. So I expect that following up the lectures and the theories with critical discussion will help students to learn better and provide them with an opportunity to improve their argumentation skills.”  


We then asked him why he applied to CATALYST; he said in reply that he would like “to access a network of people and resource in the field of social learning enabled by online technologies”. As for the core content of his proposal, it was related to the experimentation of CATALYST tools in educational application.


Luca chose Debate Hub for the experiment. He expects to learn more about the actual impact of these technologies on students learning and on how to improve the design of these platforms in order to get better outcomes. “I also expect to be able to share my ideas and the evaluation outcomes with the CATALYST network and engage into fruitful conversation and knowledge exchange”, he added.


Discover more interviews of CATALYST open-call winners such as the videos of Noemi Salantiu (EdgeRyders) and Carlos Rossique (AutoConsulta Ciudadana) on CATALYST YouTube channel and do not forget to subscribe for all live updates!


Next stop: Costa Rica – CATALYST at the FORWARD/STORY Workshop in May

FORWARD/STORY (Forward Slash Story) is an invite-only 4 days residential lab, starting from May 15-18, 2015, in Costa Rica.


The Forward/Story is a special lab for writers and designers, who work in uncharted territories to share their journey, techniques, socialize, collaborate, and solve problems. Five core challenges that storytellers face will be explored in a collaborative environment.


A special advisor at Purpose – CATALYST‘s partner, Lee-Sean Huang is one of selected storytellers. He will take CATALYST on board to the FORWARD/STORY workshop.


Find out more information about the workshop and participants at www.forwardslashstory.com, or follow on Twitter: @fwdstory




Transformative storytelling

Bring CATALYST across the ocean to Mexico City – A workshop on Transformative Storytelling

Strengthen your storytelling skills with Lee-Sean Huang,  a founding member of the design practice at Purpose, one of CATALYST’s partners.  As a part of the workshop, he will be talking about Catalyst’s Collective Intelligence Tools.


The workshop will be organized on Friday, 6th March 2015 at 9:30 am at Smart Impact/ Jardín de Innovación (an innovation Accelerator), Sonora 123. Esquina Parque España, Ciudad de México.


Find out more information about the workshop at http://jardinn.co/, and/or contact information at cowork@jardindeinnovacion.org


Stop by Smart Impact if you happen to be in Mexico City!… Nos Vemos en Mexico City!