Regulators and investors are struggling to meet the challenges posed by high-frequency trading. This ultra-fast, computerised segment of finance now accounts for most trades. HFT also contributed to the “flash crash”, the sudden, vertiginous fall in the Dow Jones Industrial Average in May 2010, according to US regulators. However, the HFT of today is very different to that of three years ago. This is because of “big data”.

监管机构和投资者疲于应对高频交易(HFT)带来的难题。这种速度极快的计算机化金融活动如今占据了交易的大头。美国监管部门认为，高频交易一手制造了2010年5月的“闪电暴跌”(flash crash)，令道琼斯工业平均指数(Dow Jones Industrial Average)突然大跌。然而，现在的高频交易已与三年前大不相同，这是“大数据”造成的。

The term describes data sets that are so large or complex (or both) that they cannot be efficiently managed with standard software. Financial markets are significant producers of big data: trades, quotes, earnings statements, consumer research reports, official statistical releases, polls, news articles, etc.

大数据指的是过于庞大或复杂（或两者兼具）、无法用标准软件高效管理的数据集。金融市场是大数据的重要产生者，交易、报价、业绩报告、消费者研究报告、官方统计数据公报、调查、新闻报道无一不是它的来源。

Companies that have relied on the first generation of HFT, where unsophisticated speed exploits price discrepancies, have had a tough few years. Profits from ultra-fast trading firms were 74 per cent lower in 2012 compared with 2009, according to Rosenblatt Securities. Being fast is not enough. We, along with Marcos Lopez de Prado of the Lawrence Berkeley National Laboratory, have argued that HFT companies increasingly rely on “strategic sequential trading”. This consists of algorithms that analyse financial big data in an attempt to recognise the footprints left by specific market participants. For example, if a mutual fund tends to execute large orders in the first second of every minute before the close, algorithms able to detect that pattern will anticipate what the fund is going to do for the rest of the session, and make the same trade. The fund will keep making the trade, with higher prices, and the algo traders cash in.

第一代高频交易单纯靠速度来发现利用价格差异，依赖这种策略的公司近年来的日子不太好过。Rosenblatt Securities表示，与2009年相比，2012年高频交易公司的利润下降了74%。光快是不够的。我们与劳伦斯伯克利国家实验室(Lawrence Berkeley National Laboratory)的马科斯•洛佩斯•德普拉多(Marcos Lopez de Prado)提出，高频交易公司越来越依赖“战略顺序交易”(strategic sequential trading)。它包含的算法可以分析金融大数据，以识别出特定市场参与者留下的足迹。例如，如果一支共同基金通常在收盘前每一分钟的第一秒执行大额订单，能够识别出这一模式的算法将预判出该基金在其余交易时段的动向，并执行相同的交易。该基金继续执行交易时将付出更高的价格，使用算法的交易商则趁机获利。

This new form of HFT can go wrong, such as in the so-called “hash crash” of April 23 2013 – the market drop caused by a bogus tweet about a terrorist attack on Barack Obama, sent from the Associated Press twitter feed. Unlike the crash of May 2010, this was not an incident caused by rapid sales triggering more sales. It was not a speed crash; it was a big data crash. Unless regulators understand the difference, they run the risk that new rules may address an old, expired challenge.

这种新形式的高频交易可能会误入歧途，例如2013年4月23日的“无厘头暴跌”(hash crash)——美联社(AP)的Twitter账号发出巴拉克•奥巴马(Barack Obama)遭遇恐怖袭击的虚假消息，引发市场下跌。与2010年5月的那次暴跌不同，此次暴跌的罪魁祸首不是快速抛售引发的更多抛售。它不是快速交易导致的暴跌，而是大数据导致的暴跌。如果监管机构认识不到区别所在，它们将面临一种风险：新制定的规则只能解决陈旧、过时的问题。

About two years ago, it became common for hedge funds to extract market sentiment from social media. The idea is to develop trading algorithms based on the millions of messages posted by users of Twitter, Facebook, chat rooms and blogs, and detect demand trends in relation to individual companies. However, these algorithms typically do a bad job when it comes to making guesses on small data sets. In recent months, it has become very popular to develop algorithms that fire off orders as soon as unscheduled information is published, such as natural disasters or terrorist attacks. More hash crash-type events, which are caused by a single erroneous data point, are disasters waiting to happen.

大约两年前，对冲基金开始普遍从社交媒体提取市场情绪信息，其理念是利用Twitter、Facebook、聊天室和博客用户发出的成百上千万条消息，开发交易算法，判断出与各家公司有关的需求趋势。然而，这些算法通常无法利用小数据集做出有效的猜测。近几个月，一种算法流行起来——一旦有自然灾害或恐怖袭击等意外信息公布，便立即抛出订单。一个数据点出错就能导致“无厘头暴跌”，能够引发它的灾难性事件未来必定会上演。

The bad news is that addressing the challenges posed by the new HFT will require understanding the mutating challenges of big data. The good news is that regulators seem to recognise the need for adaptation. This month, Scott O’Malia, Commissioner of the Commodity Futures Trading Commission, told the NYU-Poly Big Data Finance conference that “reckless behaviour” was replacing “market manipulation” as the standard for prosecuting misbehaviour. For instance, while trading on information extracted from millions of tweets is reasonable, preloading sweeping market orders as soon as an algorithm finds the words “bomb” and “White House” in the newswire is clearly reckless.

坏消息是，若要应对新型高频交易带来的难题，就必须理解大数据与以往截然不同的挑战。好消息是，监管机构似乎认识到了适应变化的必要。本月，美国商品期货交易委员会(CFTC)专员斯科特•奥马利亚(Scott O’Malia)在纽约大学理工学院(NYU-Poly)大数据金融会议上表示，“鲁莽行为”正取代“市场操纵”，成为起诉不当行为的标准。例如，尽管利用从几百万条Twitter消息提取出的信息进行交易合情合理，但一旦算法在新闻通讯中发现“炸弹”和“白宫”两词便抛出大量订单，毫无疑问是鲁莽的。

The issue at stake is, how do we make sure that participants use “Big Data” in a responsible manner? As a Harvard Business Review article put it, big data requires big judgment. A few years ago the CFTC considered whether regulators should certify traders’ algorithms. The potential for interference would be huge, not to mention the risk of intellectual property theft. A compromise may consist of market participants proposing a set of real-time indices that track “reckless” behaviour, such as adding selling pressure to a market with dwindling buyers. If a trader crosses several “recklessness” thresholds, they could be prosecuted. These indices can be adjusted and changed as markets evolve and, most importantly, they could be defined by consensus among all market participants.

一个重要的问题是，我们如何保证市场参与者负责地使用“大数据”？正如《哈佛商业评论》(Harvard Business Review)的文章所说，大数据需要大智慧。几年前，CFTC曾考虑是否应当让监管机构对交易商的算法进行认证。监管机构干预的潜在风险是巨大的，更不用说还有知识产权盗窃的风险。各方可以达成一种妥协：让市场参与者提出一系列追踪“鲁莽”行为的实时指标，例如在买家减少时增加市场抛售压力的行为。如果一家交易商逾越了多个“鲁莽”行为的临界值，它将可能被起诉。随着市场的演化，这些指标可以调整变化；最重要的是，它们可通过全体市场参与者的一致同意而制定。

One solution here is to employ the resources of the US’s National Laboratories. At the Lawrence Berkeley National Laboratory, there is the supercomputing power and analysis techniques required to monitor these “recklessness” indices in real time and advise regulators of reckless market behaviour that threatens stability. While traditional circuit-breakers halt trading after a market plunge, real-time monitoring would allow for the shutting down of individual participants, preserving the market for bona fide actors.

利用美国国家实验室的资源是一种解决方案。劳伦斯伯克利国家实验室拥有超级计算能力和雄厚的分析技术，足以实时监控这些“鲁莽行为”指标，并且向监管机构汇报威胁稳定的鲁莽市场行为。传统的停市机制在市场暴跌后停止全部交易。相比之下，实时监控能够将单个参与者扫地出门，从而向诚信的参与者继续敞开市场。

The use of big data is transforming markets. It now needs to transform how we regulate them. The solution to HFT problems is not less but more technology – and even bigger data.

大数据的使用正在改变市场。现在，我们需要改变监管市场的方式。解决高频交易问题的出路不是限制技术，而是鼓励对更复杂的技术乃至更大数据的利用。