Trading on Sentiment

How does it sound when you hear that automated algorithmic trading shall be performed based on the market sentiment analysis of your twitter updates about specific companies or stocks? Scary!!!

The potential bubble factor around such a trend is illustrated by a new hedge fund “Derwent Absolute Return” in an article by Pascal-Emmanuel Gobry in his recent article “New Hedge Fund Uses Twitter to Pick Stocks”.  Especially so when relying only on Twitter data that excludes stock specific tweets uses this information. The new fund – Derwent Absolute Return Fund from London based Derwent Capital Markets is based on a paper by researchers at Indiana University and University of Manchester. The fund has signed an exclusive deal with the researchers who published this paper, who claim that they run the twitter data through some sophisticated analysis to predict the outcome for a where the stock market was going. Here is a graphical representation of the key finding from the paper that shows the correlation between the two data series.

 

Figure 1 – the blue line shows the day to day DJIA values as compared to the red line that shows the Calm time series that predict the changes in the DJIA closing values that occur three days later

Figure 1 – the blue line shows the day to day DJIA values as compared to the red line that shows the Calm time series that predict the changes in the DJIA closing values that occur three days later

Or maybe it is not so far-fetched after all… that is if it is done the right way… to augment the inputs into the existing processes, data and algorithms to gain an additional insight (and not the only insight).

Today every financial institution gets market data feeds from various market data providers such as Reuters, Bloomberg, stock exchanges, etc. These data streams are collated and consolidated (these terms should ring a bell – Securities Master, Instrument Master, Master Data for Financial Instruments, Reference data) by the financial institutions and then provided to the “Quants” so that they can convert this market information into a trading strategy. However, given that every financial institution is getting the same market data, there is only so much you can do to slice and dice the same information in multiple ways. So how can these financial institutions have an upper hand over their competition? They have to look at new, timely and relevant sources of information that can help and aid in taking some informed decisions. This includes information from sources such as:

  • Twitter or other micro-blogging sites
  • Blogs where people talk about some more detailed analysis
  • News articles and media feeds to analyze market sentiment

Take a look at the following sentiment and trend information collated from a variety of sources and see if you spot any trends by simply looking at the lines and numbers in the individual charts.

Figure 2 – Google retail index (blue line) that tracks US only queries related to retail terms and so forth compared with Walmart stock price (WMT in red) movement on NYSE along with some other retailers (Target TGT in red, J C Penny JCP in green)

Figure 2 – Google retail index (blue line) that tracks US only queries related to retail terms and so forth compared with Walmart stock price (WMT in red) movement on NYSE along with some other retailers (Target TGT in red, J C Penny JCP in green)

Figure 3 – Sentiments Price correlation 30 days for Walmart from MktSentiment

Figure 3 – Sentiments Price correlation 30 days for Walmart from MktSentiment

Figure 4 – Real-time weighted Sentiment (green for positive and red for negative) for Walmart from MktSentiment

Figure 4 – Real-time weighted Sentiment (green for positive and red for negative) for Walmart from MktSentiment

Figure 5 – Negative sentiment analysis index rank from FlameIndex as compared to the stock volume for the last 30 days

Figure 5 – Negative sentiment analysis index rank from FlameIndex as compared to the stock volume for the last 30 days

Figure 6 – Mentions about Walmart across various sources such as blogs, microblogs, bookmarks, events, etc from SocialMention

Figure 6 – Mentions about Walmart across various sources such as blogs, microblogs, bookmarks, events, etc from SocialMention

If you did not see a distinct pattern that would allow you to start trading right away – then you are not the only one; I have been staring at this for a while now… 😉

While these samples are an illustration of the various pieces of information available, each one by itself is not an answer for sound predictive analytics in Capital Markets. In order for a financial institution to make sense of this data they have to put a holistic data strategy together that is able to:

  • convert this type of information into measurable metrics
  • correlate this information to the people who are influencers and the measure the potential impact of what they say
  • tie this information to the risk hierarchy of a given corporation for which the data is being analysed – to better understand the ripple effect of the exposure for a parent company
  • factor in the geo-political variations as a quantifiable metric that is tied to operations of a corporate entity in that region (wonder what will happen to all those egyptian cotton goods now…)
  • measure the impact of natural disasters on certain commodities and companies
Figure 7 – The various additional factors that will make the social data relevant to predicting the outcome of the market

Figure 7 – The various additional factors that will make the social data relevant to predicting the outcome of the market

Once this information is correlated, it needs to be converted into quantifiable and consumable upto the minute metrics that can then provide the various financial institutions with a better Social insight into the latest and greatest changes that are taking place in the Social sphere. These metrics can then be used as an addition data point for number crunching by the “Quants”.

Sounds great but how does one even start to solve this problem – the problem is of converting unstructured data into structured data, so that you can monitor and track the latest market signals based on information from the social sphere for a short peek into the near future.

Are we closer to a solution? The answer is a resounding YES. With the increased adoption of the Social Media (including social networks, social media, news articles, blogs and the new interaction models that capture your ratings, likes and dislikes), there is exabytes of unstructured data about everything, including people, places and stocks. In parallel the technology landscape has also changed in the last five years, which allows us to embark on a path to make sense and structure out of large amounts of data with cloud computing, NoSql databases, semantic analysis, and other related technologies. There is no silver bullet yet but the technologies are evolving to solve this problem.  Customers of Securities Master solutions and infrastructure (including users of MDM to consolidate financial instrument data) should pay special attention to this area as a potential extension of the capabilities that they have deployed in their enterprise.

Please feel free to share your thoughts and feedback.

Related articles:
Related companies used in the research for this article:
Advertisements

MDM Adoption = engaging Business Users

In a large enterprise, often an MDM implementation is a bigger political challenge than it is a technical challenge. So at every step of the MDM journey you have to choose wisely, as it is the difference between a successful or failed MDM implementation. If you are in the middle of an MDM implementation or have just completed the first phase, then you are probably thinking what should be the next step(s):

  • Should you start looking at the daunting task of laying down Data Governance policies?
  • Tackle the beast of downstream distribution to the dozen or more downstream silos?
  • Understand the ripple effect on the integrated systems of the data consolidation that takes place in the MDM system?
  • Feed the conformed dimensions into the downstream data warehouse?

Directions to choose from after you are done with phase 1 of MDM

Choose a direction for your next MDM phase

Figure 1 – which direction to choose for increasing MDM adoption across your enterprise

Your business sponsor is probably also asking (maybe not so nicely) you when will they start to see the business value out of the multi-million dollar transformation project that you spent your last six months on.

If you want to come out of this dilemma a winner, you have to focus on the ‘Adoption of MDM” data by Business Users and establish some quick wins. You need to get the eyeballs of the business users on value of the clean and mastered data, so that they can understand, appreciate and rely on the value of the value of MDM to come back and ask for more. This way you have the business as a willing and eager participant/sponsor of your MDM master plan and help generate buy-in for the next steps of the MDM roadmap in your organization.

All this sounds great in theory but how do you actually do this in practice? … hmmm… read my bestselling book, engage me as a high paid consultant… get rid of the business users… (just kidding 😉 ).

On a serious note, how do you actually come up with a practical and convincing engagement plan? Here are 3 key pointers that will help:

  1. Listen to the business and understand their pain
  2. Listen to the business and understand their pain
  3. Listen to the business and understand their pain

The Business users are the consumers of the data in question and they interact with the data on a daily basis, through the existing functional applications such as CRM, ERP, Portals, Email clients or even smartphone extensions of your enterprise apps. They rely on the information in these systems for their day-to-day business interactions (yes, even if the data is dirty or duplicate or missing). So you have to start providing them with a comparative view of what the data looks like in the MDM system, without actually embarking on a full-blown downstream integration project. The answer is to invest in what I call the “non-Intrusive” Data Integration pattern.

Non-Intrusive Data Integration

This concept is simple and a step towards enabling MDM-aware applications in your enterprise. Here are a few steps to think about that would make your applications increasingly MDM-aware and tie MDM in the daily life of your Business Users to deliver direct and immediate value:

1. Comparative MDM views – as a first step, you need to provide your users with simple and comparative MDM views, where they can not only see the data in the native application (CRM, ERP, etc) but can also launch a URL or a window to see what additional or clean information exists in the MDM system. This does not require extensive integration but simply depends on the key identifiers in the application of choice. For example, if you are using Salesforce.com (SFDC) as the CRM application, you only need the Salesforce ID for a customer record to query it from the MDM system using appropriate MDM APIs and enforcing the visibility rules similar to the ones in SFDC to show a small browser pop-up window or an inline-SFDC widget.  The Comparative views provide the users with the view of the MDM data in the application they use when they need it.  Even a simple view of the list of all the systems that are already tied into the same Account or Contact data (list of the cross-reference systems from the MDM hub) will go a long way in providing increased visibility to the benefit of MDM.

Custom actions in CRM/ERP applications to query and display MDM data

Custom action in CRM/ERP apps to query and display data from MDM

Figure 2 – provide business users the visibility to the comparative MDM data views from their preferred applications

2. MDM-augmented capabilities – typically CRM and ERP applications lack extensive history audit capabilities and hierarchy management capabilities as compared to MDM. MDM provides extensive capabilities for maintaining an audit trail of all the data changes; including the changes that take place through the MDM GUI and the changes that are fed in from batch processes or any upstream integration. In addition, MDM also provides the ability to collate, define and manage relationship and hierarchy details that are not typically present in functional applications such as CRM and ERP.  The view to these complementary pieces of information can once again be easily provided by creating either new pop-up widgets or custom screens in the functional application to provide the business users with the additional information that can be used either for compliance visibility (enterprise level audit trail of data changes for a given record) or deeper insights (multiple hierarchies. These capabilities for visibility to such data in functional business applications are also supported by some MDM vendors for certain applications out-of-the-box. Once again the focus is not on downstream synchronization but more on providing the non-Intrusive integration visibility for your Business users, without disrupting their day to day interactions.

Embedded MDM control in CRM/ERP apps

Embedded MDM control in CRM/ERP apps

Figure 3 – Embedded view of the MDM hierarchy in CRM

3. MDM-aware applications – the advanced piece of the non-Intrusive integration is to start making your functional applications MDM-aware by not only addressing the first two categories mentioned above, but also incrementally addressing pro-active data quality in your functional applications. Once the first phase of MDM is up and running with the core data (cleansed, matched and merged), you can start leveraging this data at the point-of-data-entry in functional applications. On data entry in functional applications, provide the Business Users with recommendations of potential matches that might already exist in another part of the enterprise (via MDM services and APIs), so that they can work with existing data instead of creating entirely new and duplicate data. This will help you move from reactive data correction to proactive data governance and start to ease the burden on your data governance team by reducing data duplication at the point-of-data-entry.

Enterprise user interaction channels

Enable MDM visibility across the different interaction channels for Enterprise users

Figure 4 – leverage the “non-Intrusive” MDM integration pattern across the various interaction channels for the Business Users in your enterprise

As you start investigating the “non-Intrusive” integration pattern for MDM, you also need to think about using it across all interaction channels (old and new) for the Business Users in your enterprise. It is an investment worth considering so that you can convert your Business Users from MDM skeptics to MDM champions.  This is especially true for new interaction channels that are not tied to the legacy infrastructure and applications, such as Mobile/Smart phones.

In summary, the path to MDM adoption requires you to implement the above creative and useful steps in order to deliver the master data to the ultimate consumers (the Business Users) in the shortest possible time with the least of amount of disruption.