A review of NSF funding on recommender system explanation.

Summary

I found out the explanation of recommender system is a potential research subject in a few different areas. So I did a review for the relevant project within all the NSF funding award list. Here are some of my thoughts for each project. 

• CHS: Small: How recommendation and explanation affect preferences in social networks (2014-2017)

• This project focuses on the exploration of the influence factors of recommendation and explanation on the user interaction in social media. Hence, the experiment is basically followed the existing functions in the social media services, e.g. Facebook and Twitter. This project implied the recommendation and explanation provided by social media did change the human behavior of online activity. There are always many criticisms of the social media "manipulation" the public opinion by its ranking algorithm. This concern makes this project is meaningful, due to the understanding of how the information affect user's preferences is still little known. According to the structure in my previous post, this is the first layer of the structure. 

• CHS: Small: Context-aware mobile systems to facilitate synergistic face-to-face interactions (2014-2017)

• This project related to what we are doing now of the people recommendation on CN3. I think it did require a mobile version of application to better answer the human face-to-face interaction. This would be the current trend to conduct the study. The project plans to conduct the experiment for a group of freshmen students. I think this would be a more stable user study setting for a long-term basis interaction pattern collecting. 

• CHS: Medium: Towards Transparency of Personalization on the Web (2014-2018)

• This project is very relevant to what I plan to do. They focus on the issues on sensitive data of user in the personalized system. First, they plan to distinguish the challenge for mobile user with sensitive contents. Second, develop a system with different personzied techniques to measure the prevalence. Third, identify the personalized political content. Fourth, personalized financial and health information applications. I think the PI and co-PI have a strong connection with commercial companies to gather necessary data set and user study environment. With a real world data set and system, it would be more make sense to claim the finding of privacy challenge and patterns. I wonder if there is a need for an interest-conflict-free study from my side? Say, a standalone small scale experiment to recall or further explain some of the issues can not be answered with the real world system or dataset. 

• III: Small: Technologies for Creating Explanatory and Exploratory Animations from Scientific Data (2015-2018)

• This is a project for scientific data visualization, using an animation format. Our goal is not very relevant to this one, but the idea about how them formation the data into animation and make user easy to understand or use. This could bring us some insight about how to convert the recommender system result into a user-easy-understandable format. 

• III: Small: Collaborative Research: Towards Interpretable Machine Learning (2015-2018)

• This project focuses on the decision support from machine learning, to answer how the interpretable of machine learning techniques can help the user to make a better decision. They focus on a well-known classifier - KNN as an example, to examine the interpretable in three metrics: simplicity, verifiability and accountability. The experiment is focused on how to make the classifier is intuitive to users, predictable and controllable. The similar idea can be also suitable for recommender system. But the problem is how to make the idea is novel and not just repeating the same idea from this project? 

• III: Medium: Machine Learning with Humans in the Loop (2015-2019)

• A new direction of integrating human behavior to machine learning algorithm. This project is aimed to better facilitate the human behavior with the design of machine learning algorithm. There are some latest publications start to answer the issues and challenge in this area of "Humans in the Loop". This project is also focusing on the decision process and support from human. Furthermore, to design a better interface to connect the learning algorithm and human behavior. Ultimately, as a human interactive learning system. I am wondering the connection between human in the loop and the human in the "user modeling". In recommender study, we model the user based on their preferences and historical data, there is few studies discuss about how to let user to join and understand the process. 

• CAREER: Analyzing Interactions in Visual Analytics for User and Data Modeling (2015-2020)

• I think this project answers my concern in "III: Medium: Machine Learning with Humans in the Loop". This project intends to develop an interface to extract users' high level knowledge, to a better user or data modeling. The visual interface can help to better analyze the human-machine interaction. As a long term grant, this implied some of the potential in this direction of research. But I think the visual analytic approach is only one of the way to engage user into the system. In some of the cases, for example, the text mining, the visual tool may not that useful. There may be some further potential work I can pursue in recommender system study. 

• CHS: Medium: Collaborative Research: Beyond the Black Box: Understanding and Designing for User Expectations of Algorithmic Media (2016-2019)

• This project discussed more about the public awareness of the widely used algorithms. This is the latest funding since July 2016, which means, the state of the arts of the current researches. According to my literature review, this is right on the spot of the most potential research topic in this area. What I want to do on recommender system is pretty similar to this one. The main difference is that, I focus on more about the recommender system to item or people, but this project emphasizes more on the social media. However, I agree, based on the social media would be more suitable or simply to respond some of the interesting issues or channledge across disciplines. For instance, the law and ethics of the post ranking on Facebook or Twitter to affect the user political leanings. Even so, I think the recommender system can answer this question in a more genelize perspective, say, how the transparency and help answer the media bias. Also, the more potential issues in different area, e.g. e-commerce or location-based people system. 

(*Rank by start-end funding year.)

Explainable Artificial Intelligence Systems

In a military information system of training or tactics, an after-action-review(AAR) is the most common use approach to learn from the exercise. With the complexity of artificial intelligence system, it is hard for user to interact or question the outcome from an "AI-controlled system". This feature caused the following challenges: 1) the user is hard to understand how the result is made or processed; 2) the user feedback is not well considered by the system; 3) the situation is hard to re-produce for training or debug purpose, the users need to re-run the system until the certain criteria occur. In [1], the research proposed a user interface for a military simulator system. The user can "ask question" by subject, time and entity. 

However, the user interface can only provide the "straightforward" information for users. For example, during a simulator, the user can ask "What is your location/health condition/current task"? All of these are only the attribution in the system that's not difficult to retrieve and display. In today, with the data mining and machine techniques, many of the attribution are lacking of a straightforward way to explain it. For instance, a decision made by targeting system with a deep, multiple layer neunor network, with a hundred rounds or training and testing. In this case, to provide the explain of why choose A instead B would be a more challenge issue. 

For national security and military purpose, this issue is even more critical in following aspects: 1) for training purpose: if the users have no idea how the system works, it is impossible for users to interact or even correct the wrong decision make by the system or algorithm; 2) accountability: if the system made a wrong decision, it is hard to account the responsibility between human and machine; 3) security issue: if all the data process and analysis are in a block box, there is a security concern to use the technique in the real world environment - no one knows if the system was hacked or wormed. 

The military system seems far away from us, but actually, the similar issue has been discussed for personalized system in [2]. There are several main issues for a personalized system without a "scrutinize and control": privacy, invisibility, error correction, compatibility across systems and controllability. It seems to have an overlapping between the two research directions. In personalized system, the research focus on the intractability of the user modeling to help with the system effective, trust, transparency and comprehensive. In explainable artificial intelligence systems, it more focus on AAR for military purpose. For example, in [3], it provided another case of how the exploitation of AAR help users in the medical training session, for self-evaluation and problem-solving. The explainable AI system plays an educational role for training purpose. 

Either of the two directions, plus the state-of-the art machine learning techniques would be a great research subject. Here is the a note for three layers, machine learning categories: 

• Layer 1: Classifier (Supervised)

• ADA-Boost, Logistic regression, SVM, Knn, Naive Bayes, decision tree: The classification method is basically trying to find out a point, line or faces split the 2 to N type of elements, based on the feeding training/testing data
• The issue here is we need to extract the “features” from the raw data, rather than a set of raw data. The features should reflect the original data property as possible.
• It would be simple to show the feature in different latent space. For instance, to show a regression line to distinguish the classification question. 
• Layer 2: Markov Chain (semi-Supervised)
• Hidden Markov Model (HMM): based on a series of decision process, to find something unknown.
• In Markov model, we need to define the motion as a sequential state, a series of observations. The model is trained for maximum the output probability.
• In [4], the author dealt with the similar issues for a neuron network decision process (control and interaction), a self-explainable approach is still unknown. 
• Layer 3: Deep learning (Unsupervised)
• Convolutional neural network (CNN), Recurrent Neural Network (RNN), Deep Neural Network (DNN): automatically extract features from convolutional, recurrent or deep strategies, plus the method above to train/test the models.
• In the two approaches above, the human needs to extract the features based on some inference. For example, a concept from physics. These features are interpreted by some prior-knowledge. What if in some of the cases, the feature extraction is almost impossible? E.g. The image recognition.
• In this layer, many of the features are not recognizable. In face, we can use "eigen-face" to visualize the image recognition features. For the other domain, it is hard to visualize the features. Furthermore, the state-of-the-art approach combines the classifier in layer 1 and the feature extraction in layer 3. It remains many challenge research topics in algorithm, interface and human perception. 

Reference

1. Core, Mark G, H Chad Lane, Michael Van Lent, Dave Gomboc, Steve Solomon, and Milton Rosenberg. “Building Explainable Artificial Intelligence Systems.” In Proceedings of the National Conference on Artificial Intelligence, 21:1766. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.
2. Kay, Judy, and Bob Kummerfeld. “Creating Personalized Systems That People Can Scrutinize and Control: Drivers, Principles and Experience.” ACM Transactions on Interactive Intelligent Systems (TiiS) 2, no. 4 (2012): 24.
3. Lane, H. Chad, et al. Explainable artificial intelligence for training and tutoring. UNIVERSITY OF SOUTHERN CALIFORNIA MARINA DEL REY CA INST FOR CREATIVE TECHNOLOGIES, 2005.
4. Kim, Been. Interactive and interpretable machine learning models for human machine collaboration. Diss. Massachusetts Institute of Technology, 2015.

Thoughts on Exposure to ideologically diverse news and opinion on Facebook

Summary

The ranking algorithm on the social media is always a controversial issue across disciplines researchers. For instance, the famous debate of filter bubble and echo chamber effect. The state-of-the-art data mining and machine learning techniques, actually, enforcing the phenomena. Due the Facebook ranking algorithm become smarter and smarter, your post wall is filled with all the content you preferred, lack of diversity or multiple voices. Even worse, manipulated by some of the commercial or private purpose.

In this paper that Facebook published in Science magazine, it is the first time to response this issue based on the real world massive data set. The finding are: 1) the stronger ideological alignment will come with higher share numbers. In other words, the article with strong perspective would be re-post more from users, and not a surprise, by the same alignment of users (i.e. Liberal users will tend to re-post more liberal articles and vise versa); 2) the homophily of the friends. The users with the similar ideological affiliation will tend to friend each other on Facebook. The data analysis reflects a clear pattern that the liberal and conservative both with less friend ties of different ideological affiliation, in other words, less diversity. 3) The crosscutting percentage is dropping when the content explosure decreasing. More specifically, when the user can randomly browse all the content on Facebook post, there are around 40-45% of crosscutting rate. However, the rate dropping dramatically, when the user selects from within friend circle, algorithmic suggestion or by themselves. More interestingly, the paper makes a conclusion that, the lower diversity reads/share behavior is mainly due to the individuals' choices.

This is valuable research due to this is the first time to reveal the detail pattern from the Facebook real-world data set. However, I against the conclusion they made, also thinking about the other potential research topic that we can pursue. Here are my reasons: 1) to account the responsibility for the user is not fair due to most of the users, they, have no clue about how the algorithm behind the system will affect their future information consumption. For example, the Facebook ranking algorithm will penalty the ranking score if you not to "like" or "share" the content you saw, say, the news articles. Hence, the news article you ignore will, slowly, disappear on your wall post. And, the mechanism is not transparent at all. The user will never know some of the content they are pre-filtered, due to some of the ignorance action they did. I argue if the "ignore" or "don't like" represent the preference of dislike for each user? 2) There is no way for the user to understand or join in the loop of algorithice processing. The user is basically followed the system suggestion or guidelines, in a very "user-friendly" and "simple" way. There should be either a way to explain or "undo" function let user can maintain the diversity content consumption, in their own will. Also, a double reminder when you decide to unfollow or dislike something, does a reminder interface require, just like when you decide to permanently delete a file from your computer? 3) The user deserves the controllability. Why there is only one personalized ranking algorithm for all different kinds of users? I think the user has the right to choose the preference they like, rather than, decided by some unknown experts or machine learning algorithms. I think this would be fair to claim, the less diverse is due to the click behavior by the users.

The three points above are the potential research topics in my point of view. If we think about Facebook ranking in a recommender system, the same discrimination or less diverse issue may also happen just under our nose. Furthermore, the potential conflict of interest the researcher from industry should be revealed. I admin the researcher from big companies with more resources to answer some of the social phenomena than a laboratory environment. For example, the Google trend for flu prediction and the Facebook ideologically diverse across real-world system users. However, the commercial companies are responsible to their stock holder, not to publics. I believe this should be the advantage for researcher in academia to play a natural role on the research subjects. This is also the value to establish a small scale system and seek for controlling experiments.

1. Bakshy, Eytan, Solomon Messing, and Lada A. Adamic. "Exposure to ideologically diverse news and opinion on Facebook." Science 348.6239 (2015): 1130-1132. APA

Summary of Interactive and interpretable machine learning models for human machine collaboration

Summary

I found this thesis is very relevant to what I want to do in my dissertation study. This paper basically considers to fill up the gap between human and machine collaboration, based on the approach of interactive and interpretable models. More specifically, this thesis develops a framework of "human-in-the-loop machine learning" system. There are three parts to consist this paper: 1) build up a generative model for re-produce human decision process. This section is aimed to extract the relevant information from a natural human decision process, to prove the machine learning can effectively predict the humans' sequential plan; 2) a case-based reasoning and prototype classification. This section is aimed to provide a meaningful explanation to user to better engage with the system. The goal of this part is to examine the interpretability of the proposed system; 3) an interactive Bayesian case model. This section focus on an interactive Bayesian model the human or expert can contribute their knowledge or preference into the system. The author used a graphical user interface for the interaction between humans and machine, in an online educational system.

Although the thesis structure is very similar to what I want to do. However, the author focus on more in the Bayesian decision support model. All the findings and systems are based on the model. Say, how a rescue team to form up a resource allocation plan, within a sequential decision step? How the domain expert can contribute their knowledge into the model to generate a better result (model accuracy)? How the graphical interface can help user to input their feedback and get a better model result? But in my expertise, we should think about this issue in a different perspective: recommender system.

If I follow the same three layer structure, the whole idea would be: First, interaction patterns, to understand the human behavior to the machine, e.g. The recommender system with a complex machine learning or data mining techniques. I need to know how human interaction with the system and how the system can help them better fulfill the task they care about. More specifically, to test how the interpretability/transparency/explain functions are required for human to interact with the recommender system. Second, an effective system, to design a system that can help user to retrieve the useful information or suggestions. So based on the finding of the above, we can design a novel system to implement the functions we found, e.g. What kind of interpretability/transparency/explain functions that really help users to better use the recommender system. Third, a human-in-a-loop model, based on the above two findings, we know the functions are necessary and useful. Here comes a challenge to make the human can involve in the process. More specifically, an interactive recommender system that support user to contribute their preference or domain knowledge to improve the system or user experience.

1. Kim, Been. Interactive and interpretable machine learning models for human machine collaboration. Diss. Massachusetts Institute of Technology, 2015.
2. Explain of recommender system: a literature review.

Summary of Hypertext/UMAP 2016 conference.

Summary

This conferene was combined with UMAP2016 and Hypertext2016.

I had a short paper presentation in this conference. The topic is about personalized recommender system for local businesses using Yelp dataset [1]. It was nice to have some feedback from the audience. Here are some of the paragraphs: 1) if I defined the business relationship based on the user review, how can I make sure the sequential shopping behavior? Say, the user from one restaurant to another ice cream shop? I think this suggestion is critical. But I pre-filter the data into daily basis, i.e. The same day shopping patterns. So this may cover the most sequential between any two businesses share the same group of users; 2) two attendees asked about the data pre-processing issue. They wonder how I can make sure its a sequential shopping for any two of the businesses? This is related to the previous question. I pre-filter them into a daily basis; 3) the long distance shopping pattern between Las Vegas and Phoenix. Some of the audience likes the idea to see the commercial pattern across cities; 4) for future study, the system of this paper requires a user study of the costumers. I may send out the questionnaires to the customers in different businesses, to see if the recommendation result fits their shopping preferences. Besides, my presentation, I talked to many of the attendees at the conference. I believe it would be a meaningful connection for future collaboration.

1. Tsai, Chun-Hua. "A Fuzzy-Based Personalized Recommender System for Local Businesses." Proceedings of the 27th ACM Conference on Hypertext and Social Media. ACM, 2016. APA

Summary about the 2016 big scholarly workshop (WWW companion)

Summary for 2016 Big Scholar Workshop

This is a great event for scholarly data oriented researchers to share the ideas and interact with each other. The talk from this workshop is very informative from several scholars with rich academic reputation. The keynote speaker included: Dr. C. Lee Giles (Pennsylvania State University), Dr. Jie Tang (Tsinghua University). Dr. Giles shared his works on CiteSeerX system and the future plan about open and share the dataset to other researchers. He also mentioned some of the interesting work, e.g. To estimate the scholarly document number on the web. Dr. Tang shared about his works on Aminer. This is another famous project about the scholarly portal and recommendation system. His talk is mainly about the detail of system implementation, e.g. How to collect paper and parse them into structured text, how to maintain the user profile for data accuracy, how to find out the domain expert based on collected scholarly data, etc. The Aminer system is now focusing on the expert recommend task. I raised one question about how many percentages the user actually maintains/interact with the system. Dr. Tang replied: although everyone can edit/modify any profiles, but few users are actually using the system. To make sure the data accuracy, they will focus on the manually maintain on some of the listed domain experts.

There are some more famous scholars joined the workshop, included: Dr. Jevin West (University of Washington), Dr. Feng Xia (Dalian University of Technology), Dr. Huan Liu (Arizona State University), Dr. Kuansan Wang (Microsoft Research) and Dr. Philip S. Yu (University of Illinois at Chicago). It is nice to hear their presentation and have some feedback on my work from them. Their feedback included: 1) why not included venue information for the prediction model? 2) Can you predict the future junior school productivity based on your model? 3) How do you sample the postive/negative size for the model evaluation? This is a critical point for the performance of the classification problem. 4) How do you define the junior scholar age? All of the feedback is valuable for me to refine my future works. [1]

Since there are many projects work on the scholarly data analysis, e.g. Google Scholar, Microsoft Academic Search, CiteSeerX, Aminer and more (Conference Navigator). The closing remark is discussing the platform to utilize the data in different sources and create this as a community for scholarly data researches. The research topic can be extended into data sciences, education, health and more. Dr. Giles and Dr. Wang is actually initiated the next workshop or conference into a broader scope. Dr. Wang, as a representative from industry, is agreeing to provide some of the infrastructure support for all scholars to work on the big scholarly projects. Dr. Giles is also willing to open the dataset for further collaborations. I believe this would be a potential research direction for future studies.

1. Tsai, C. H. and Lin, Y.-R. (2016), Tracing and Predicting Collaboration for Junior Scholars. WWW 2016 Proceedings (workshop paper)

Bringing social networks into a physical space - social sensing computing

Summary 

This paper used the RFID technology to recognize the user's social cluster from DBLP database. The authors provided a big display screen within a conference that participants can interact with the interpersonal connections. This paper actually consisted with several interesting elements together: 1) the RFID sensor: the RFID sensor is aim to recognize the user approaching the display screen and provide the personified social network graph to user; 2) graph exploration: the display screen provided the zoom in/out function that provide the exploration functions for users to discover the interpersonal relationship; 3) heterogeneous network: the displayed graph contained with the conference/co-citation/co-authorship features, this make the user can discover the hidden relationship or knowledge insides the network.

The experiment result showed a positive feedback from users, but some of the issues attract my eye: 1) limited user usage: only few of the conference participants actually used the system, fewer people used more than one time; 2) the cold start problem: some junior scholar may not have publication or their publication is not listed inside the DBLP database; 3) the privacy issue that display your personal social network in a public view display screen; 4) the cost and purpose to deploy RFID: the usage of the RFID tag is not significantly necessary.

This idea is interesting to make the conference with more fun and entourage people to explore the social interaction during the event. But I think this tool should be more personalize and privacied. For instance, to display the result with a personal device, e.g. cell phone. In other way, the design of the graph is not highlighting the meaningful information for users. To show everything on screen is almost equal nothing for users. This might be the reason the user return rate is low. These issues would be valuable for the future application.

Besides, the RFID tag implied a research subject about the social sensing computing. With the developing of wearable devices, there are more and more sensor, tag or device that with the potential to load the user data for multiply computing task. For research purpose, to gather all of these data is critical, also, difficult. However, there are many alternative way to "simulate" the scenarios. For instance, a QR code, a reference number or mobile phone built-in function (e.g. GPS, Bluetooth). All of the technologies are interesting, but also require many developing efforts. Nor sure if this could be our expertise to do all these things by our own?

Reference

1. Konomi, Shini'chi, et al. "Supporting colocated interactions using RFID and social network displays." Pervasive Computing, IEEE 5.3 (2006): 48-56. Link

Summary for two papers of facilitate the conference by social games

Summary 

To help conference attendees gain social capital or expand social networking is always an interesting research topic. One of the direction is by "social games". The paper from [1] and [2], both design a social game to facilitate people social interaction inside a conference. In  [1], the author designed a game requires 2-6 people to communicate on a puzzle of ball and hole location matching. This is a tool for "ice-breaking" between the conference attendees through the teamwork procedure. In [2], the author discussed an approach to gain community retention by the social game. They build up a cell phone app that supports a collaborative function on the task. The users can solve problems insides the game with others, as a community. The author argued, this app encourages user to improve their network connectivity. Furthermore, this app may be used inside the class to increase the class retention rate for minority groups.

Both of the papers proposed an interesting idea about social game. However, I think the evaluation would be an issue to support the above claims. In [1], the author proposed a questionnaire for the game players. Based on the response, the users indicated the communication and team-work function are the most important election for them. The evidence to support the game usefulness in helping users to "making friends fast" is not strong. In [2], the authors only present the evaluation plan about how the social game is helping to build a community. The experimental data are still lacking.

I think the research question can be more specific classified as 1) cold-breaking; 2) social interaction and engagement; 3) social recommendation; 4)social networking ; 5) teamwork and communication and 6) community formation and retention. For each of them, the experimental design should be varied. Some of the aspect is hard to find a ground truth to prove the model/game/app effectiveness. For instance, if the user talk to each other more due to the apps? It is not easy to compare the talk frequency before/after the game play. Hence, an experiment design for certain research questions is critical. Some of the ideas: 1) A/B testing to different group of users; 2) quick questionnaire/feedback insides the game; 3) clicking/bookmarking/friendship behavior analysis, etc.

Reference

[1] Evie Powell, Rachel Brinkman, Tiffany Barnes, and Veronica Catete. 2012. Table tilt: making friends fast. In Proceedings of the International Conference on the Foundations of Digital Games(FDG '12). ACM, New York, NY, USA, 242-245. DOI=10.1145/2282338.2282386 http://doi.acm.org/10.1145/2282338.2282386

[2] Samantha L. Finkelstein, Eve Powell, Andrew Hicks, Katelyn Doran, Sandhya Rani Charugulla, and Tiffany Barnes. 2010. SNAG: using social networking games to increase student retention in computer science. In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education (ITiCSE '10). ACM, New York, NY, USA, 142-146. DOI=http://dx.doi.org/10.1145/1822090.1822131

ExcUseMe: Asking Users to Help in Item Cold-Start Recommendations

Summary

This paper is mainly about the solution of item cold start problem in the real world recommender system. In an on-line recommender system, the cold start problem is caused by lacking of the historical data to generate meaningful suggestions. In other words, the cold start problem can be treated as the user or item that newly enter into the system. In general, the cold start problem was solved by include new context or content features. However, in this paper, the author focus on the specific solution for collaborative filtering (CF). Their idea is conducting a small scale user experiment to those users who might interested in the new item. The user interaction within the experiment can be used as a reference for the new item recommendation.

The major challenge behind this idea is: Who is the target user? In this paper, they proposed a novel algorithm "ExcUseMe". The authors assumed the user is randomly visited the online recommender system. In this continuing user stream, the system required to determine who should be included in the new item experiment. The approach can be divided into two stages: 1) the learning phase; 2) the selection phase. In 1), the algorithm is computing the likelihood of the user provide feedback of the item. In stage 2), the user is selected by their likelihood ranking. The vector similarity between the users are also considered in a semi-greedy manner.

The experiment simulated the real world on-line setting, they sampled the n% users as selection pool. The performance was evaluated by RMSE (Root-mean-square deviation) metric for three large scale datasets. The baseline was compared with Random, Frequent Users, Distance and Anava approaches. The experiment result indicated the proposed ExeUseMe algorithm outperform all the baseline models, moreover, with a lower computational cost. The experiment result also pointed out the importance and contribution of positive feedback from users. In other words, the key to drive this method as a meaningful output is the user participants.

This paper highlight a new approach for CF to solve the cold start problem. Besides, this approach is also practical that can be adopted into the real world on-line system. The main contributions of this paper are 1) the real-world user selection method; 2) efficiency algorithm and 3) the high model performance in most of the scenarios. There are some comments in this paper: 1) this algorithm is valuable in an on-line experiment setting, but the experiment is simulated with off-line setting and dataset. A further simulation experiment can better demonstrate the advantage of this approach; 2) the data filtering section limited the user with between 20 - 300 ratings. This setting reduced the probability to provide useful suggestions for long-tail items; 3) In real-world recommender system, the small amount of user feedback lead to the result of a recommendation. There might be some incentive to inverse control the experiment result in some ways.

Reference

Michal Aharon, Oren Anava, Noa Avigdor-Elgrabli, Dana Drachsler-Cohen, Shahar Golan, and Oren Somekh. 2015. ExcUseMe: Asking Users to Help in Item Cold-Start Recommendations. InProceedings of the 9th ACM Conference on Recommender Systems (RecSys '15). ACM, New York, NY, USA, 83-90. DOI=http://dx.doi.org/10.1145/2792838.2800183

Relescope: An Experiment in Accelerating Relationships

Reading Summary

To help people to digest the information is always an interesting research question. In this paper, the author tried to produce a 1-2 page short report to the conference attendees based on their previous works. This report provided a meaningful relationship network of the conference and aim to better determine the suitable social activity during the conference, for example, to recognize and talked to people. The author sends out the questionnaire to examine the validity of the report. The survey showed the effeteness of this application. Moreover, the newcomer of the conference is benefited more than the senior participants.

This implication reminds a fundamental way of human to digest information: the "device" to provide personalized information to users. The paper is published in the year 2005. At that time, since the cell phone is not popular with the public yet, the author sends out the application result by paper. Sometimes, the small piece of paper might be useful and convenient for people to access and carry the information during the conference. This is the reason, until today, the paper handout is still the most necessity in an academic conference.

The paper-based approach is with the natural limitation - update the information and interaction. Besides, it is also hard to collect user behavior or feedback based on this approach. With the most popular on mobile and wireless technology, it is possible for people to stay online all day. The social media, e.g. Facebook, Twitter and Linkedin, start to grab the attention of participants of the physical environment. People are now interacting in physical and virtual. The exploration of social community behavior is a popular research topic today. For instance, the human behavior/interaction on Twitter during a conference.

The future study might be focused on the personalized recommendation, e.g. Conference Navigator, tried to provide a personalized application interface for conference participants. However, there is still an unanswered question about how to help the newcomer of the conference to better into the environment and to their future academic career. More precisely, to promote a meaningful social connections. The way to evaluate the effectiveness of social connections is a challenge here.

Reference:

1. Farrell, S., Campbell, C., and Myagmar, S. (2005) Relescope: an experiment in accelerating relationships. In:  Proceedings of CHI '05 extended abstracts on Human factors in computing systems, Portland, OR, USA, ACM, pp. 1363-1366, also available athttp://dx.doi.org/10.1145/1056808.1056917.

A talk summary of "Professional Research Opportunities for Ph.D. Students in MSR"

Summary 

This is a talk summary of [1], the speaker mentioned:

1. It is a new era of Microsoft research with new CEO and board member. The research lab now more follows and support the three pillars of the new company agenda, compare to the time of bill gate. There are three pillars of MSR (Microsoft Research Lab): Productivity, Smart cloud and Personal computing, e.g. health care. The mission statement is, like academics to pursue knowledge, looking for new ideas and innovation from MSR to the corporation. 
2. Two constraint of applying a funding for researcher: the budget from the Congress is shrinking, also the current political situation in Congress: more young assistant that might lack of the passion on sciences and more focus on sweet issues, e.g. education bills and subsidy. 
3. The difference between the research lab and academia: in MSR, you don't need to worry about the research funding for graduate students, you have colleagues and interns. You will get 12 months salary and the research lab inspires the fundamental research questions. It is a bottom up style; In academia, you will need more writing proposals, sometimes it is hard to get money to support your research. 
4. MSR cares the career professional development. Their own career. Welcome to co-work with other teams. Seek to impact to the world. Get your knowledge or innovation on the Microsoft product. Get things into practical products. 
5. In big-data era, there is an advantage to work in research lab: the massive data inside the company that cannot share outside of the world. This would be benefited with the research that doing speech, translation, machine learning and some more research subjects. 
6. Microsoft also puts things out of open source. Not like IBM, who much more value patents, but MSR respects more to academic value. When you interview a position, what is the cultural difference in there? This is a question worth to ask. 
7. MSR evaluated the research by their impact. They expected you as the expert in some fields, also bring value to companies. Change the peer-review rule two years ago, not to only counting your publication. No tenure track. 
8. MSR also encourage to deliver the research to start up. Welcome that kind of people to bring in more different genes. 
9. No matter you are in industry or academia, you will learn how to expose your work in confidence.  Who does your audience you focus on? E.g. how to explain your work to WSJ reporter? This is a story telling absolute you need to learn. Also, to get funding is important in academic, good to know who control the faucet (money). 

Thoughts

I feel the goal of Microsoft from the new board of directors is clear: productivity. All the products and services from MS will be around this core. The MSR would not be an exception. The vice-president speech clearly points out the three pillars of the mission statement. It is hard to avoid the demanding from boards about delivery the research, innovation to the value of the company. In another way, this might be a chance to bring the research into a real world products or services. However, is the mission goal of company will crowd out the support of fundamental researches? This might be an another question worth to ask. Besides, to leverage the huge dataset from the MS products and services would really be a beneficial for researchers, e.g. the social network from Hotmail, the behavior patterns from Office 365 or the user generated data from Windows. All the data is valuable for researcher to proceed the experiments. This might be also a disadvantage and challenging for data sciences researchers in academia. Any thoughts?

Reference

1. Professional Research Opportunities for Ph.D. Students, Jeannette M. Wing, http://halley.exp.sis.pitt.edu/comet/presentColloquium.do?col_id=8982

Technology policy research of Korea - mobile platform and network neutrality

Summary

The technology policy from the government always plays a critical role in the industry. In the research of [1], the authors intended to measure the efficiency difference before and after the platform standardization policy formulation and implementation (WIPI). They adopted the idea of "efficiency frontier"to measure the mobile company performance difference. According to their empirical finding, the government-led mobile platform standardization policy is negatively affecting the mobile companies' efficient, compare to the company independent to the mobile network operators. The author suggested the government should play a supporting-role in the policy regulation.

In paper [2], the author discussed the network neutrality effects on new internet application services. They proposed a simulation experiment to examine the service diffusion in different network regulation settings. They found the "more latency sensitivity and broader bandwidth services have displayed a higher willingness to pay (WTP) for high-priority Internet services" which accordance to with the assertion of network provider have the incentive to charge additional fees on certain services. However, they further explored the diffusion effect under government regulation. They found the discrimination from network provider might hurt the growth of new-coming internet service diffusion. They also suggested the government need to take in action to protect the new innovative service in early stage.

Thoughts

The research of technology researches is full of the regional differences. However, this is a good way to reference the experience of other countries. For the first paper, the authors imply an interesting research question: what is the role that government should play in the new mobile era? The technology is rapidly changing time by time. It is pretty hard for a government to propose a complete and flexible policy for the new industrial business. The idea of WIPI is clear, the Korean government intends to construct a universal mobile platform inside Korea. If the regulation succeeds, they can have a giant eco-system to promote their mobile application industry, as a game rule maker. The idea is similar to Japan mobile platform that adopted more localize special specifications. However, even larger market in Japan can not resist the platform competence from Apple ios and Google Android. That is why the government-led standard regulation is hard to compete with the other two industries-led platforms. There are more economic issues behind the scene, this could be another interesting research subject.

The dispute of network neutrality is another debate between network service providers and operator. There is a famous case from Netflix, a high latency sensitivity and broader bandwidth on-line video streaming services. The story is ended in Netflix pays the extra access fee to network operator Comcast. This action is a revenue worth decision for Netflix, but might be a huge barrier for some new internet service, as the paper [2] claims. The FCC is trying to forbid the discrimination charge from network operators, but this rule also arouses great controversy on the government internet control issue. The multiple stakeholder in this game makes this dispute continual. The simulation approach of this paper would be a way for us to examine the internet control policy.

Reference:

1. Hongbum Kim, Daeho Lee, Junseok Hwang. (2016). Measuring the Efficiency of Standardisation Policy Using Meta-Frontier Analysis: A Case of Mobile Platform Standardisation, International Journal of Mobile Communications.
2. Lee, Daeho, and Hongbum Kim. "The effects of network neutrality on the diffusion of new Internet application services." Telematics and Informatics 31.3 (2014): 386-396.

Reading Summary: "Content Driven User Profiling for Comment-Worthy Recommendations of News and Blog Articles"

Summary

The main research goal of this paper [1] is to recommend a "Comment-Worthy" article list for users. The authors argued the previous research [2] adopted a content-based collaborative filter approach that lack of the consideration of the relation between article segments and comments. They provided an example about cell phone introduction news, the comments listed below might connect to the different segments inside the article, or even the irreverent interests from readers. Hence, it is inappropriate to consider an article and its comments as a single topic. They proposed a Collaborative Correspondence Topic Models (CCTM) approach that can capture the probability distribution between article-comment and article-user, and follow by the Monte Carlo Simulation, Gibbs Sampling, stochastic MCEM algorithm to estimate the latent offsets and generated the perdition list. According to the paper, the experiment indicated the performance is better than three baseline models in both cold-start and warm-start settings basis. 

Thoughts

The idea to consider the relation between article-comment and article-user is interesting. The authors take the user profiling (based on the user comments) into account, to build up a user-personalized comment-worthy recommendation list. This paper provided a way to link the user feedback and their user interests. This could be a way to build up the user profiling (modeling). The idea in this paper could be applied to the person's recommendation as well. However, I have a couple comments on this paper: 1) the empirical result pointed a clean dataset difference. The model performance of the Daily Mail is much better than the other two. There might be some characteristic exist in the dataset that make the model can do a better prediction task; 2) the commenting behavior is diversity. Many of the comments are actually junk posts. I think an exploration of the dataset comment distribution would be better to understand the commenting behavior; 3) the proposed is adopted content-based and topic model approaches. We can only see a slight improvement between the new proposed model and the CoTM (pure content-based approach). 

Some possible potential research topics: 

• Inline commenting behavior: what is the user replying to? For any real world events or the articles, news, social network posts. There are tons of the re-ply posts from the public readers or their social network friends. It will be interesting to see the re-play behavior and analysis in different scenarios. For example, when a new apple product announcement news published, what is the comment mainly about? Or, when an emotional (happy, sad, exciting, etc.) tweets showed on Twitter, what is the response from their following/followers? 
• Content Driven User Modeling: we might take the published text or other user generated content to build up a user model. This model could apply to recommendation task, behavior comparison, performance analysis, etc. It will be interesting if we can leverage the current information to the other targets, e.g. to solve the cold-start problem or learn from the other rich data source. 

More...

• The idea about use sampling to inference the data distribution from machine learning perspective. E.g. the Markov chain Monte Carlo (MCMC), Gibbs sampling. 
• Topic modeling approaches, e.g. CORR-LDA, Collaborative Topic Modeling

Reference

1. Bansal, T., Das, M., & Bhattacharyya, C. (2015, September). Content Driven User Profiling for Comment-Worthy Recommendations of News and Blog Articles. In Proceedings of the 9th ACM Conference on Recommender Systems (pp. 195-202). ACM.
2. Shmueli, E., Kagian, A., Koren, Y., & Lempel, R. (2012, April). Care to comment?: recommendations for commenting on news stories. In Proceedings of the 21st international conference on World Wide Web (pp. 429-438). ACM. ISO 690