Tag Archives: search

Menczer comments on real-time search in IEEE Spectrum

realtimesearchThe IEEE Spectrum piece Real-Time Search Stumbles Out of the Gate discusses the recent integration of real-time search features, such as Twitter and other microblog entries, into major search engines. Professor Filippo Menczer, CNetS associate director, comments in the article on the challenges posed by real-time search. Here is an excerpt of the interview:

IU’s Menczer suggests that with all this user-generated content, the environment is more complex than the one Google’s PageRank algorithm had to deal with. While search used to be about relationships between pages, he explains, now it’s about relationships between ”people, tags, Web pages, ratings, votes, and direct social links….It may not be that page A points to page B but rather that user John follows Mary and replies to the tweet of Jane and retweets it.” That makes it ”a more complicated ecosystem,” he says, ”but a very rich one,” and search engines will need ”more sophisticated ways to extract data from these relationships” […]

Read more…

Bias

The aim of this project is to characterize, study and model various sources of bias that emerge from the complex network structure of the Web, social media, and search engines.

Social bias

social bubblesSocial media have become a prevalent channel to access information, spread ideas, and influence opinions. However, it has been suggested that social and algorithmic filtering may cause exposure to less diverse points of view. In the paper Measuring Online Social Bubbles we quantitatively measure this kind of social bias at the collective level by mining a massive datasets of web clicks. Our analysis shows that collectively, people access information from a significantly narrower spectrum of sources through social media and email, compared to a search baseline. The significance of this finding for individual exposure is revealed by investigating the relationship between the diversity of information sources experienced by users at both the collective and individual levels in two datasets where individual users can be analyzed—Twitter posts and search logs. There is a strong correlation between collective and individual diversity, supporting the notion that when we use social media we find ourselves inside “social bubbles.” Our results could lead to a deeper understanding of how technology biases our exposure to new information. A release about this work got some press coverage and an extended version of this paper is in preparation.

Gender bias in wikipedia

Contributing to the writing of history has never been as easy as it is today. Anyone with access to the Web is able to play a part on Wikipedia, an open and free encyclopedia, and arguably one of the primary sources of knowledge on the Web. In our paper First Women, Second Sex: Gender Bias in Wikipedia we study gender bias in Wikipedia in terms of how women and men are characterized in their biographies. To do so, we analyze biographical content in three aspects: meta-data, language, and network structure. Our results show that, indeed, there are differences in characterization and structure. Some of these differences are reflected from the off-line world documented by Wikipedia, but other differences can be attributed to gender bias in Wikipedia content. We contextualize these differences in social theory and discuss their implications for Wikipedia policy. This work was covered in Wikimedia Research Newsletter. An extended journal version titled Women through the glass ceiling: gender asymmetries in Wikipedia also shows that women in Wikipedia are more notable than men, which we interpret as the outcome of a subtle glass ceiling effect.

Popularity bias in search

bias

The feedback loops between users searching information, users creating content, and the ranking algorithms of search engines that mediate between them, lead to surprising results. We are studying how all these systems and communities influence and feed on each other in a dynamic information ecology, and how these interactions affect their evolution and their impact on the global processes of information discovery, retrieval, and utilization.

For example, studying the relationship between Web traffic and PageRank, we have shown that given the heterogeneity of topical interests expressed by search queries, search engines mitigate the popularity bias generated by the rich-get-richer structure of the Web graph. These results, dispelling the feared Googlearchy affect, have been published in Proc. Natl. Acad. Sci. USA, presented at the WAW 2006 keynote (slides), and generated some media attention. You can see some movies demonstrating the finding. The result also inspired a robust rank-based model of scale-free network growth, published in Phys. Rev. Lett. (press release).

We also study sources of bias that stem from legal, political, or economic factors. The CENSEARCHIP tool visualizes the differences between results obtained from different search engines, or different country versions of a search engine. This tool, based on a technique described in this paper in First Monday, generated a lot of reactions in the media and the blogosphere (press release).

Project Participants (some of them)

Fil Menczer, PI
Fil Menczer
Sandro Flammini
Sandro Flammini
Alex Vespignani
Alex Vespignani
Santo Fortunato
Santo Fortunato
Mark Meiss
Mark Meiss
Dimitar Nikolov
Dimitar Nikolov

Support

Pervasive Technology Labs at Indiana University Mark Meiss is supported by the Advanced Network Management Laboratory, which is one of the Pervasive Technology Labs established at Indiana University with the assistance of the Lilly Endowment.
Volkswagen Foundation Santo Fortunato was supported by a Volkswagen Foundation grant.
Nsf_logo This research is also supported in part by the National Science Foundation under awards 0348940, 0513650, and 0705676.

Opinions, findings, conclusions, recommendations or points of view of this group are those of the authors and do not necessarily represent the official position of the National Science Foundation, the Volkswagen Foundation, or Indiana University.

Sixearch

The goal of Sixearch (carl.cs.indiana.edu/6S) is to provide an open-source platform for developing a context aware personalized peer-to-peer (P2P) distributed information retrieval system. The application currently supports collaborative Web search with scalability.

Sixearch ArchitectureSixearch uses the idea of modeling neighbor nodes by their content but without assuming the presence of special directory hubs. As shown on the left, each peer is both a (limited) directory hub and a content provider; it has its own topical crawler guided by its user’s information content and local search engine. Peers communication is built on JXTA platform. When a user submits a query, it is first matched against the local engine, and then routed to neighbor peers to obtain more results. Ideally, the peer network should lead to the emergence of a clustered topology by intelligent collaboration between the peers. While traditional search engines such as Google and Yahoo provide access to very large document collections, the Sixearch P2P Web search application provides a complementary way for users to actively and collaboratively share their own document collections. However, the Sixearch framework allows traditional search engines to naturally be included as peers; such peers would quickly emerge as reliable, trustworthy, and general authority nodes.

A screenshot of peer interactions taken from the network visualization applet
A screenshot of peer interactions

The right figure displays a screenshot of the queries being sent among peers. Peer interactions are visualized by an applet. The area of each node is proportional to the size of its Web index. The edges represent the queries exchanged between two peers. The connectivity of each peer is an indirect measure of centrality, authority, and/or reliability of the peer as learned by the other peers.

Our work on Sixearch has been published in AAAI Magazine (preprint), and presented at Hyperterxt 2009 (demo), ACM SAC2009 (paper), RIAO2007 (demo), ACM CIKM P2PIR2006 (paper), WTAS2005 (paper), WWW2005 (poster), and WWW2004 (poster). Unfortunately, we have also been the victims of shameless plagiarism.

Visit Sixearch to learn more and download the application or contribute to it!

Members & Collaborators

Fil Menczer, PI
Fil Menczer
Le-Shin Wu
Le-Shin Wu
Ruj Akavipat's picture
Ruj Akavipat
Namrata Lele
Namrata Lele
Rossano Schifanella
Rossano Schifanella
Ana Maguitman
Ana Maguitman

Support

Nsf_logo Sixearch project is based upon work supported by the the National Science Foundation under award No. IIS-0133124 and IIS-0348940 CAREER: Scalable Search Engines Via Adaptive Topic-Driven Crawlers.
ibm Recently, Sixearch project has received the IBM 2007 UIMA award on 6S: A Collaborative Web Search Network.

NaN

News | People | Publications | Meetings | Mailing list | Talks | Pics | Computing | Data | Coffee

Networks & agents Network

NaN is a research group exploring the modeling, simulation, and analysis of complex social and information networks, adaptive agents, and social computing systems. We especially focus on social media and the Web as complex techno-social networks in which we leave abundant traces of our activities: what we do, what we are interested in, whom we talk to, what knowledge we acquire and contribute. Our research spans from modeling the dynamic processes that occur online (how information networks grow and evolve, how they can be manipulated for the spread of misinformation, how individual and collective traffic patterns emerge, how attention bursts are generated and shaped by social and search tools) to designing tools that mine the Web to build better search, navigation, management, and recommendation tools (where ‘better’ means more reliable, intelligent, autonomous, robust, personalized, contextual, scalable, adaptive, and so on). We have ongoing collaborations with colleagues at the IU Network Science Institute (IUNI)ISI Foundation, Yahoo Labs, and MoBS Lab.

Active NaN projects

OSoMe
Observatory on Social Media (OSoMe)
Algorithmic Bias
Algorithmic Bias
Attention and Virality
Attention and Virality
Kinsey Reporter
Kinsey Reporter
DESPIC
DESPIC
Scholarometer
Scholarometer
Web Dynamics
Web Dynamics
Web Traffic Analysis & Modeling
Web Traffic
Web Security
Web Security

Archived NaN projects

GiveALink
GiveALink
Text & Link Modeling
Text & Link Modeling
Sixearch
Sixearch
Network Flow Analysis
Network Flow Analysis
WebGraph++
WebGraph++
InfoSpiders
InfoSpiders
Web Topologies
Web Topologies
IntelliShopper
IntelliShopper
ELSA
ELSA
LEE
LEE
BioNets
BioNets
ACE
ACE

Talks: Torino, Padova, Genova, Roma

This sabbatical is providing wonderful opportunities for me to present our work and establish/strengthen collaborations with several groups in Italy. Recently I have given invited seminars on social search at the Department of Informatics at the University of Torino (hosts Matteo Sereno and Mino Anglano) and on Web traffic at the Department of Math at the University of Padova (host Massimo Marchiori). In the next few weeks I will give a talk on social search at the Department of Informatics and Information Science at the University of Genova (host Marina Ribaudo) and one on search engine bias and Web modeling at my old stomping ground, the Institute of Cognitive Sciences and Technologies of the National Research Council in Rome (host my undergraduate advisor and mentor Domenico Parisi).

Visit to Yahoo! Research

I just got back from a visit to Yahoo! Research Silicon Valley. I gave two talks presenting our work on social search and web traffic analysis, and met lots of interesting people. They have an amazing group and of course mountains of data to lust after. Hopefully this will lead to collaborations in the future, given the many intersecting research interests.

4 talks at ECCS 2007

We are giving 4 invited talks at the European Conference on Complex Systems, in Dresden (October 1-5, 2008):
  1. Web click traffic network (by Fil) at Complex Networks: Dynamics and Topology Interplay