Como criar um sistema de comércio de alta freqüência

O Guia do Heretic para Finanças Globais: Hacking the Future of Money.
Aventuras Anárquicas em Artes, Ativismo, Antropologia e Economia Alternativa.
Quarta-feira, 17 de junho de 2015.
Surrealismo algorítmico: um guia lento para negociação de alta freqüência.
PARTE 1 (3500 palavras)
Um primário de 900 milhões de microsecondos sobre o comércio de alta freqüência.
No tempo que leva você a ler esta frase, um algoritmo de negociação de alta freqüência (HFT), conectado a uma bolsa de valores via & # 8220; baixa latência e # 8221; infra-estrutura comercial, poderia fazer, talvez, mil transações.
1.1: colocando HFT em contexto.
Houve um tempo, no passado distante da década de 1970, quando os negócios nas bolsas de valores eram basicamente o domínio exclusivo dos atores humanos. Se era o investidor prudente e de longo prazo comprando uma carteira de ações para um fundo de aposentadoria, ou o especulador de cowboy comprando e vendendo em rápida sucessão, o processo sempre foi limitado pela velocidade da mente humana e o tempo necessário para realmente pegue um telefone e faça um pedido. Mesmo o especulador mais rápido ainda demoraria vários minutos para completar os negócios.
1.2: Como devo me sentir sobre isso?
1.3: Negociação, para negociação técnica, para trocas comerciais, para HFT.
Deixe recuar um passo atrás e tente colocar essa atividade em contexto. Os mercados financeiros como uma bolsa de valores facilitam a compra e venda de instrumentos financeiros, que são contratos que lhe dão direitos para receber retornos ao longo do tempo. Eles tendem a hospedar diferentes jogadores com diferentes horizontes temporais. Nos anéis externos, você obtém os grandes investidores institucionais, como os fundos de pensão. Eles chegam no mercado de vez em quando e fazem grandes investimentos, comprando um grande número de ações, muitas vezes com vistas a mantê-los por vários anos. Então, nos anéis internos, você fica mais rápido, mais inconstante, jogadores e # 8211; podemos chamá-los de comerciantes & # 8211; que ganham dinheiro saltando dentro e fora dos mercados, como tubarões ágeis nadando entre as vagens mais lentas de grandes baleias.
Comece por entender o conceito geral de negociação: os comerciantes financeiros compram e vendem instrumentos financeiros, como ações em empresas. Eles esperam comprar a um preço mais baixo do que vendem, obtendo assim lucro. Agora entenda o comércio técnico: os comerciantes têm diferentes técnicas de especulação. Eles podem, por exemplo, passar horas investigando os registros de uma determinada empresa para fazer avaliações, uma prática chamada de negociação fundamental. Alternativamente, eles podem analisar as atividades de outros comerciantes em um mercado para tomar decisões. Esta "análise técnica" de preços, pedidos e dados de volume gerados por outros comerciantes leva a negociação técnica. Agora imagine que seja automatizado no Algorithmic Trading: pode-se decidir automatizar o processo de negociação técnica, de modo que um algoritmo analise um fluxo de dados de preços, pedidos e volumes e faça negócios sob certas condições. Chamamos isso de negociação algorítmica. (nota: é possível fazer uma distinção entre negociação algorítmica e automatizada, mas, para facilitar, vamos assumir que estas são as mesmas) Então imagine que acelerou o processo de negociação de alta freqüência: se você acelerar esse processo de negociação algorítmica automatizada a velocidades extremas, você está fazendo operações de alta freqüência. O HFT é, portanto, o melhor inicialmente pensado como uma negociação algorítmica muito rápida, que por si só é uma negociação técnica automatizada, que por si só é um sub-ramo de negociação mais ampla. Pode-se contrastar com, por exemplo, uma negociação mais lenta e fundamental, o que as pessoas gostam de George Soros (ele e seus analistas se sentam em uma sala e observam o mundo e fazem grandes apostas sobre isso). Finalmente, lembre-se de que podemos novamente contrastar esse mundo inteiro de negociação com o mundo do investimento de longo prazo, que é o que os fundos de pensão grandes e lentos fazem. Para retornar à analogia anterior do ecossistema, então, as empresas HFT são como piranhas entre os tubarões entre as baleias.
1.4: Como configurar uma empresa HFT.
Diferentes organizações comerciais podem ter razões ligeiramente diferentes para se envolver em HFT. Alguns grandes bancos, por exemplo, usam-no como uma ferramenta para tomar uma grande ordem e fragmentá-lo em muitos pequenos pedidos, como usar um bocal de dispersão para transformar um jato de mangueira de fogo em uma névoa de mercado fina que as pessoas não aviso fácil. Muitos jogadores de HFT, porém, são puramente especuladores de curto prazo, firmas comerciais especializadas e hedge funds. Se você quisesse definir um desses, aqui estão algumas coisas que você faz.
Este não é um artigo técnico pedante sobre a natureza exata da tecnologia HFT. Existem enormes quantidades de discussões de bumf carregadas de jargão e geeky na internet, se você realmente está interessado no técnico, mas a essência do que você tem a fazer é: você deve cortar algum tipo de acordo com uma corretora firme e uma bolsa de valores para obter o seu fantástico algoritmo o mais próximo possível da bolsa de valores! Você deve minimizar a distância física entre o computador em que seu algoritmo está inserido e o computador em que o sistema de correspondência de pedidos da troca está dentro, de modo que os dois possam entrar em um diálogo intenso e de velocidade leve entre si.
1.5: Aperfeiçoe a arte eletrônica da guerra.
Agora, não são essas empresas, todas usam as mesmas estratégias. Alguns usam análise estatística e arbitragem de vários tipos, enquanto outros operam exclusivamente na "microestrutura de mercado" # 8221; estratégias, que parecem envolver o conhecimento das astúcias eletrônicas íntimas dos sistemas de intercâmbio e como eles podem ser, um, aproveitados. Pode-se envolver no comércio flash, que alguns argumentam é uma forma de legalização em frente. Você pode comercializar os mercados com ordens através do & # 8221; pedir recheio & # 8243; (O que o HFT denomina Dave Lauer chama um ataque financeiro DDOS).
& # 8220; A maioria dos [HFT] é um jogo de velocidade muito simples de arbitragem. Ambos são dinheiro ou mercados de futuros ou em ações que eles são atingidos / recolhidos em um ECN e vendem / compram em outro lugar, a totalidade ou a maior parte do dinheiro é feita a partir de uma fração de descontos em ações de mercado. Algumas poucas empresas fazem milissegundos de negociação de impulso, eles percebem que alguém está entrando com ordens e eles pisam as ordens (porque são mais rápidos para chegar ao mercado) eles empurram o mercado um centavo e vendem de volta para o comprador original e # 8230; Eles também usam ordens flash para saltar para frente de grandes pedidos. Alguns também examinam a profundidade do livro e tentam trocar também. Existem algumas estratégias mais que utilizam em ações. Além disso, algumas empresas consideram os mercados de opções e os arbustos dota. A maioria das estratégias não são matemáticas, mas relacionadas à microestrutura dos mercados e # 8230; Essas lojas são lojas de alta freqüência, pode haver até milhões de pedidos por dia dependendo de quantos mercados e quão ativamente comercializam. Eles exigem principalmente conjuntos de habilidades C ++ realmente bons, conhecimento de conectividade API do lado do software. O lado do hardware requer um conhecimento de hardware de nível muito baixo, como ignorar a pilha e os kernels enganadores. Eles também procuram garotos lan / wan que podem enviar dados de alguns microseconds mais rapidamente na rede. Eles usam equipamentos muito caros e especializados. Um interruptor simples que é decentemente rápido custa 50K & # 8230; Todos os dados que estão disponíveis para grupos HF estão disponíveis para todos os comerciantes, a diferença é que eles trocam essa informação antes que você possa mesmo recebê-la em seu computador. Quão rápido eles podem obtê-lo e reagir no mercado é a diferença. Eles estão lidando com latências de microssegundo de um dígito em suas redes e computadores, não em milissegundos. & # 8221;
1.6: O (estreito) debate acadêmico.
Longe de todas as explicações de vídeo do Youtube, relatórios jornalísticos e discussões de fórum sobre HFT, obviamente existe um conjunto de pesquisas acadêmicas. Se você estiver procurando por argumentos robustos ao invés de Quumumar & # 8217; s gunslinging & # 8220; & # 8217; deixe-me dizer-lhe como é & # 8221; as pessoas conscientes da pesquisa, o indivíduo orientado para pesquisa pode navegar nas revistas acadêmicas. Há pesquisas emergentes de departamentos de finanças e economia, sem surpresa, mas também de algumas outras disciplinas.
1.7: Pesquisa e lobby, informa um debate regulatório.
Muitas das notícias sobre HFT são as batalhas políticas e as ameaças dos reguladores para reprimir isso. Teoricamente, o debate regulamentar deve ser informado pela pesquisa acadêmica, mas é claro que também podemos suspeitar que os debates regulatórios sejam igualmente informados pelo lobby.
PARTE 2 (4500 palavras)
Cinco quadros para ver o HFT.
2.1: o progresso imparável de agentes racionais sem agência.
Independentemente dos apelos à moralidade da HFT, há outra linha mais sutil. Observe o nome do grupo de lobby pro-HFT acima mencionado # 8211; A Iniciativa de Mercados Modernos. O nome é uma tentativa deliberada de pintar qualquer pessoa preocupada com HFT como inimiga do progresso moderno, no caminho do inevitável triunfo de um mundo mais eficiente e racional. O dogma de tecnologia como progresso está amplamente arraigado em nossa sociedade, como um pedaço de malware social difícil de remover que destrói os impulsos críticos das pessoas. O impulso "luddita" é ridicularizado, em vez de celebrado como um ceticismo saudável em relação às ferramentas dos poderosos.
Essa visão do agente-racional-sem-agência é algo que impede um pensamento muito importante sobre a economia, uma mistura estranha de exaltar a virtude do indivíduo que toma riscos e, simultaneamente, afirmando que eles são irrelevantes, meros fantoches que atuam vontade do mercado # 8217 ;.
2.2: Automação: as pessoas criam robôs para crunch (grandes) dados.
Para ser um dos agentes econômicos acima mencionados no espaço HFT, você precisa dominar três coisas. Em primeiro lugar, você precisa dominar o hardware físico: os cabos reais, cabos e torres de microondas. Então, você deve poder dominar os fluxos de dados que viajam através desses fios, para coletá-lo e organizá-lo de forma eficiente. Então você deve ser mestre do algoritmo que sabe o que fazer com base nesses dados. O algoritmo é o seu avatar automatizado no mercado, & # 8216; pensando & # 8217; e agindo em seu nome.
2.3: Automação: os robôs criam dados (grandes) para atrapalhar as pessoas.
Nós tendemos a entender o conceito de ativamente usar a tecnologia para alcançar certos fins (agência de exercício), mas achamos mais difícil conceituar a perda potencial de agência que a tecnologia pode trazer. É um fenômeno talvez melhor demonstrado com o e-mail: eu posso usar o email para exercer minha agência neste mundo, para enviar mensagens que fazem as coisas acontecerem. Ao mesmo tempo, não é como se eu realmente tivesse a opção de não usar o email. Na verdade, se eu não tivesse uma conta de e-mail, ficaria gravemente desativado. Há uma contradição em jogo: o e-mail me capacita, ao mesmo tempo me ameaçando com falta de poder se eu me recusar a usá-lo.
2.4: Meninos desconectados com brinquedos perigosos.
Tendo desviado as possibilidades de um surgimento emergente das máquinas, podemos voltar para a Terra e olhar para o mundo humano da HFT. Quem são as pessoas individuais envolvidas, e quais são as dinâmicas culturais?
O mercado de ações é difícil. Não nos deve nada. Ele punha nossos erros. Outros têm mais dinheiro, mais poder, mais conexões. Nós somos perdedores. Continuamos aprendendo. Inovamos. Todos os dias é uma nova luta. A tecnologia é nossa arma. Nós fazemos milhões de pequenos negócios. Nós reduzimos as perdas. Identificamos oportunidades. Nós focamos. O mercado pode ser espancado. Nós amamos o jogo.
Ao avaliar essas declarações de combate banal-como-mortal-combate, eu gosto de usar o Teste de Avô da Segunda Guerra Mundial, o que envolve-me perguntando o que meu avô diria sobre isso. Ele foi um piloto de bombardeiros durante a Segunda Guerra Mundial e foi derrubado sobre a Alemanha, aterrando uma pilha de metal flamejante no litoral depois provavelmente matando muitas pessoas com bombas incendiárias. Pergunte ao seu avô: o que você acha da batalha do Tradebot com o & # 8216; mercado?
2.5: HFT e a financiarização do ruído sem sentido.
& # 8216; Financialisation & # 8217; é um termo carregado de várias interpretações, mas tende a referir-se à crescente importância do setor financeiro na vida econômica geral, à infusão de normas e moralidade do setor financeiro na cultura cotidiana e ao processo pelo qual as coisas anteriormente não combinadas se tornaram financeiras produtos que podem ser negociados nos mercados financeiros.
Possível significa que o assunto pode ser reivindicado por alguém e que eles podem excluir outros de seu uso. & # 8216; Ownability & # 8217; confia em ser capaz de isolar e separar algo das coisas ao seu redor. Por exemplo, o movimento do gabinete envolveu transformar terra em parcelas demarcadas que poderiam ser separadas uma da outra e Investable de propriedade privada significa transformar a coisa em um bem que entrega retornos ao longo do tempo. Enquanto um pedaço de terra pode ser algo que você pode possuir, e ter uma conexão emocional também, você pode começar a vê-lo como um "bem" quando é usado para produzir rendimentos ao longo do tempo. Pode ser percebido como um "investimento" genérico, ao invés de um pedaço de terra com uma história e vida particular. Tradível significa que esse recurso pode ser repassado a outros.
Eu possuo uma fazenda. Eu posso usá-lo para fazer comida eu tenho uma participação na fazenda do meu vizinho. Eu posso reivindicar uma parte do produto que eu possuo uma participação em uma pequena empresa agrícola privada. Recebo dividendos monetários anuais e leio os relatórios que possuo uma participação em uma grande empresa agrícola de capital aberto. Recebo dividendos monetários e posso vender minhas ações para outros em qualquer ponto da bolsa de valores Eu possuo uma participação em um enorme fundo negociado em bolsa de agricultura (ETF) que detém ações de empresas agrícolas em todo o mundo Eu possuo uma participação em uma cobertura Fundar que comercializa rapidamente uma carteira de tais ETFs e aposta em tais ETFs por meio de derivativos.
Ambas as técnicas dependem de um tipo de sensibilidade, de uma consciência de alguma realidade externa e de uma capacidade de raciocinar sobre isso. No caso do comércio fundamental, é a conscientização sobre um novo desenvolvimento no mundo do petróleo. No caso do comércio técnico, é a consciência de uma nova tendência que se desenvolve entre outros comerciantes.
No encerramento: surrealismo algorítmico parasitário.
Um problema comum em pensar sobre a HFT, porém, é que as pessoas afastaram-se com elas. Eles se sentem em pânico por quão alienígenas parece, tendo visões do colapso do mercado extremo como algoritmos desonesto executar tudo em uma orgia gigante psicodélica de roteadores.
Para mim, porém, a pergunta realmente interessante sobre HFT não é essa fixação banal sobre a interrupção ou não dos mercados. São os elementos culturais e políticos. É assim que uma coisa tão ridícula pode ser considerada legítima. E, é a pura fisicalidade, o fato de que parece "ephermeral" ainda confia em uma enorme infra-estrutura do mundo real para se envolver na atividade essencialmente sem sentido. E, é a geografia. É um conjunto de tecnologia que tenta eliminar distância e tempo, mas a percepção de distância e tempo são dois componentes principais de uma sensação de diferença entre os lugares. Elimine a sensação de distância e o tempo necessário para chegar lá, e você pode criar a ilusão homogeneizante de estar em vários lugares simultaneamente. A interface do computador em uma empresa HFT global, presidida por múltiplos mercados globais, é um agente de homogeneização suave.
Leitura adicional: as pessoas devem seguir informações atualizadas sobre HFT.
E finalmente.
Essas peças me levam idades para escrever, então, se você quiser apoiar minha escrita Creative Commons, por favor considere me comprar uma cerveja virtual. Felicidades!
16 comentários:
Oi Brett, essa é uma peça bastante extensa, deixando-me com uma sensação de por que diabos eu incomodo, mesmo tentando trocar. O Big Short de Michael Lewis foi o grande choque, uma boa leitura para o quão sujo esse negócio realmente é.
Sim, confira seu livro Flash Boys também, neste tópico específico. Felicidades.
O pouco engraçado sobre esta afirmação: "Quero dizer, eu quero dizer, engenheiros aeroespaciais fazem algo que é bastante complexo e não posso dizer-lhe como eles fazem isso tecnicamente, mas eu entendo o que eles fazem em princípio"
Eu tenho certeza que eles fazem o Mason. Meu ponto aqui não era muito se os computadores estão ou não envolvidos na engenharia aeroespacial, mas sim o fato de ter uma compreensão intuitiva do objetivo final da engenharia aeroespacial (que muitas vezes não temos com a HFT)
Todo comércio tem duas partes dispostas a negociar e em retrospecto sempre terá um vencedor e um perdedor. Portanto, a HFT poderia ganhar um monte de dinheiro ou perder muito muito rapidamente. Como um trader chamado, estou chocado porque nunca mencionou isso.
Não tenho certeza por que isso é inteiramente relevante, e me desculpe por ter ficado chocado, mas com certeza. A peça não diz que você não pode perder dinheiro com HFT.
Quando troco meu padeiro um pão em troca de dinheiro, quem é o vencedor e quem é o perdedor?
Você é um escritor fantástico, e essa foi uma leitura realmente interessante. Obrigado.
Esse é o tipo de você para dizer Ben, cheers!
Este comentário foi removido pelo autor.
Bem escrito e você faz alguns pontos de reflexão no final.
Melhor introdução do comércio de alta freqüência que eu já li. Grande artigo ... Paul.
Grande peça, obrigado!
Qual é a duração padrão das opções cambiais negociadas na Plataforma de Negociação de Alta Freqüência da Dow Jones.

Fundamentos do comércio algorítmico: conceitos e exemplos.
Um algoritmo é um conjunto específico de instruções claramente definidas destinadas a realizar uma tarefa ou processo.
O comércio algorítmico (negociação automatizada, negociação em caixa preta ou simplesmente algo-trading) é o processo de uso de computadores programados para seguir um conjunto definido de instruções para colocar um comércio para gerar lucros a uma velocidade e freqüência impossíveis para um comerciante humano. Os conjuntos definidos de regras são baseados em tempo, preço, quantidade ou qualquer modelo matemático. Além das oportunidades de lucro para o comerciante, o algo-trading torna os mercados mais líquidos e torna a negociação mais sistemática descartando impactos emocionais humanos nas atividades comerciais. (Para mais, consulte Picking the Right Algorithmic Trading Software.)
Suponha que um comerciante siga esses critérios de comércio simples:
Compre 50 ações de uma ação quando sua média móvel de 50 dias excede a média móvel de 200 dias. Vende ações da ação quando sua média móvel de 50 dias está abaixo da média móvel de 200 dias.
Usando este conjunto de duas instruções simples, é fácil escrever um programa de computador que monitorará automaticamente o preço das ações (e os indicadores de média móvel) e colocará as ordens de compra e venda quando as condições definidas forem atendidas. O comerciante não precisa mais manter um relógio para preços e gráficos ao vivo, ou colocar as ordens manualmente. O sistema de negociação algorítmica automaticamente faz isso para ele, identificando corretamente a oportunidade comercial. (Para mais informações sobre as médias móveis, consulte Médias móveis simples, faça as tendências se destacarem.)
[Se você quiser saber mais sobre as estratégias comprovadas e pontuais que podem eventualmente ser trabalhadas em um sistema de comércio alorítico, confira o Curso de Torneio de Dia de Torneio da Invastopedia Academy. ]
Benefícios da negociação algorítmica.
A Algo-trading oferece os seguintes benefícios:
Negociações executadas com os melhores preços Posicionamento instantâneo e preciso da ordem comercial (com altas chances de execução nos níveis desejados) Negociações cronometradas corretamente e instantaneamente, para evitar mudanças de preços significativas Custos de transação reduzidos (veja o exemplo de falta de implementação abaixo) Verificações automatizadas simultâneas em múltiplos condições de mercado Reduziu o risco de erros manuais na colocação dos negócios Backtest o algoritmo, com base nos dados históricos e em tempo real disponíveis Reduzida a possibilidade de erros por comerciantes humanos com base em fatores emocionais e psicológicos.
A maior parte do dia-a-dia é a negociação de alta freqüência (HFT), que tenta capitalizar a colocação de um grande número de pedidos em velocidades muito rápidas em múltiplos mercados e múltiplos parâmetros de decisão, com base em instruções pré-programadas. (Para obter mais informações sobre o comércio de alta freqüência, consulte Estratégias e Segredos de Empresas de Negociação de Alta Freqüência (HFT).)
O Algo-trading é usado em muitas formas de atividades de comércio e investimento, incluindo:
Investidores de médio a longo prazo ou empresas de compra (fundos de pensão, fundos de investimento, companhias de seguros) que adquirem ações em grandes quantidades, mas não querem influenciar os preços das ações com investimentos discretos e de grande porte. Os comerciantes de curto prazo e os participantes do lado da venda (fabricantes de mercado, especuladores e arbitragentes) se beneficiam da execução comercial automatizada; Além disso, ajudas de algo-trading na criação de liquidez suficiente para os vendedores no mercado. Os comerciantes sistemáticos (seguidores de tendências, comerciantes de pares, hedge funds, etc.) acham muito mais eficiente programar suas regras comerciais e permitir que o programa seja comercializado automaticamente.
O comércio algorítmico proporciona uma abordagem mais sistemática ao comércio ativo do que os métodos baseados na intuição ou instinto do comerciante humano.
Estratégias de negociação algorítmica.
Qualquer estratégia de negociação algorítmica exige uma oportunidade identificada que seja rentável em termos de melhoria de ganhos ou redução de custos. As seguintes são estratégias de negociação comuns usadas em algo-trading:
As estratégias de negociação algorítmicas mais comuns seguem as tendências em médias móveis, fuga de canais, movimentos no nível de preços e indicadores técnicos relacionados. Estas são as estratégias mais fáceis e simples de implementar através de negociação algorítmica porque essas estratégias não envolvem fazer previsões ou previsões de preços. Os negócios são iniciados com base na ocorrência de tendências desejáveis, que são fáceis e direitas de implementar através de algoritmos sem entrar na complexidade da análise preditiva. O exemplo acima mencionado de média móvel de 50 e 200 dias é uma tendência popular seguindo a estratégia. (Para mais informações sobre as estratégias de negociação de tendências, consulte: Estratégias simples para capitalizar as tendências.)
Comprar um estoque cotado duplo a um preço mais baixo em um mercado e simultaneamente vendê-lo a um preço mais alto em outro mercado oferece o diferencial de preço como lucro ou arbitragem sem risco. A mesma operação pode ser replicada para ações versus instrumentos de futuros, pois os diferenciais de preços existem de tempos em tempos. Implementar um algoritmo para identificar esses diferenciais de preços e colocar as ordens permite oportunidades lucrativas de forma eficiente.
Os fundos do índice definiram períodos de reequilíbrio para que suas participações fossem compatíveis com seus respectivos índices de referência. Isso cria oportunidades rentáveis ​​para comerciantes algorítmicos, que capitalizam os negócios esperados que oferecem lucros de 20 a 80 pontos base, dependendo do número de ações no fundo do índice, apenas antes do reequilíbrio do fundo do índice. Essas negociações são iniciadas através de sistemas de negociação algorítmica para execução atempada e melhores preços.
Muitos modelos matemáticos comprovados, como a estratégia de negociação neutra do delta, que permitem a negociação de combinações de opções e sua segurança subjacente, onde os negócios são colocados para compensar deltas positivos e negativos, de modo que o portfólio delta seja mantido em zero.
A estratégia de reversão média baseia-se na ideia de que os preços altos e baixos de um bem são um fenômeno temporário que retorna periodicamente ao seu valor médio. Identificar e definir uma faixa de preço e implementar algoritmos com base em isso permite que os negócios sejam colocados automaticamente quando o preço do recurso entra e sai do seu alcance definido.
A estratégia de preços médios ponderados por volume quebra uma grande ordem e libera pedaços menores determinados dinamicamente da ordem para o mercado usando perfis de volume histórico específicos de estoque. O objetivo é executar a ordem perto do preço médio ponderado do volume (VWAP), beneficiando assim o preço médio.
A estratégia de preço médio ponderado no tempo quebra uma grande ordem e libera dinamicamente determinados pedaços menores da ordem para o mercado usando intervalos de tempo uniformemente divididos entre o início e o fim do tempo. O objetivo é executar a ordem perto do preço médio entre os horários de início e término, minimizando assim o impacto no mercado.
Até que a ordem comercial seja totalmente preenchida, este algoritmo continua enviando ordens parciais, de acordo com o índice de participação definido e de acordo com o volume negociado nos mercados. A "estratégia de etapas" relacionada envia ordens a uma porcentagem definida pelo usuário de volumes de mercado e aumenta ou diminui essa taxa de participação quando o preço da ação atinge os níveis definidos pelo usuário.
A estratégia de falta de implementação visa minimizar o custo de execução de uma ordem através da negociação do mercado em tempo real, economizando assim o custo da ordem e beneficiando do custo de oportunidade da execução atrasada. A estratégia aumentará a taxa de participação direcionada quando o preço das ações se mover de forma favorável e diminuí-lo quando o preço das ações se mover de forma adversa.
Existem algumas classes especiais de algoritmos que tentam identificar "acontecimentos" do outro lado. Esses "algoritmos de sniffing", usados, por exemplo, por um market maker market market têm a inteligência interna para identificar a existência de qualquer algoritmo no lado da compra de uma grande ordem. Essa detecção através de algoritmos ajudará o fabricante de mercado a identificar grandes oportunidades de ordem e permitir que ele se beneficie ao preencher os pedidos a um preço mais alto. Isso às vezes é identificado como front-running de alta tecnologia. (Para obter mais informações sobre negociação de alta freqüência e práticas fraudulentas, consulte: Se você comprar ações on-line, você está envolvido em HFTs.)
Requisitos técnicos para negociação algorítmica.
Implementar o algoritmo usando um programa de computador é a última parte, batida com backtesting. O desafio é transformar a estratégia identificada em um processo informatizado integrado que tenha acesso a uma conta de negociação para fazer pedidos. São necessários os seguintes:
Conhecimento de programação de computador para programar a estratégia de negociação necessária, programadores contratados ou software de negociação pré-fabricado Conectividade de rede e acesso a plataformas de negociação para colocar os pedidos Acesso a feeds de dados de mercado que serão monitorados pelo algoritmo para oportunidades de colocar pedidos A capacidade e infra-estrutura para voltar a testar o sistema uma vez construído, antes de entrar em operação em mercados reais Dados históricos disponíveis para backtesting, dependendo da complexidade das regras implementadas no algoritmo.
Aqui está um exemplo abrangente: o Royal Dutch Shell (RDS) está listado na Amsterdam Stock Exchange (AEX) e London Stock Exchange (LSE). Vamos construir um algoritmo para identificar oportunidades de arbitragem. Aqui estão algumas observações interessantes:
AEX negocia em Euros, enquanto a LSE negocia em libras esterlinas. Devido à diferença horária de uma hora, a AEX abre uma hora antes da LSE, seguido de ambas as trocas comerciais simultaneamente durante as próximas horas e depois de negociar apenas na LSE durante a última hora à medida que o AEX fecha .
Podemos explorar a possibilidade de negociação de arbitragem nas ações da Royal Dutch Shell listadas nesses dois mercados em duas moedas diferentes?
Um programa de computador que pode ler os preços atuais do mercado Os feeds de preços de LSE e AEX A taxa de câmbio para a taxa de câmbio GBP-EUR Capacidade de colocação de pedidos que podem rotear a ordem para a troca correta do recurso Back-testing em feeds históricos de preços.
O programa de computador deve executar o seguinte:
Leia o preço de entrada do estoque RDS de ambas as bolsas Usando as taxas de câmbio disponíveis, converta o preço de uma moeda para outra. Se houver uma discrepância de preço suficientemente grande (descontando os custos de corretagem) levando a uma oportunidade rentável, então coloque a compra ordem em troca de preços mais baixos e ordem de venda em troca de preços mais elevados Se as ordens forem executadas conforme desejado, o lucro de arbitragem seguirá.
Simples e fácil! No entanto, a prática de negociação algorítmica não é simples de manter e executar. Lembre-se, se você pode colocar um comércio gerado por algo, os outros participantes do mercado podem também. Conseqüentemente, os preços flutuam em milissegundos e até mesmo em microssegundos. No exemplo acima, o que acontece se o seu comércio de compras for executado, mas o comércio de vendas não acontece à medida que os preços de venda mudam quando o seu pedido atinge o mercado? Você vai acabar sentado com uma posição aberta, tornando sua estratégia de arbitragem inútil.
Existem riscos e desafios adicionais: por exemplo, riscos de falha do sistema, erros de conectividade de rede, atrasos de tempo entre ordens comerciais e execução e, o mais importante de tudo, algoritmos imperfeitos. O algoritmo mais complexo é o backtesting mais rigoroso antes de ser posto em ação.
The Bottom Line.
A análise quantitativa do desempenho de um algoritmo desempenha um papel importante e deve ser examinada criticamente. É excitante ir pela automação auxiliada por computadores com a noção de ganhar dinheiro sem esforço. Mas é preciso certificar-se de que o sistema está completamente testado e os limites exigidos são definidos. Os comerciantes analíticos devem considerar a aprendizagem de sistemas de programação e construção por conta própria, ter confiança em implementar as estratégias certas de forma infalível. O uso cauteloso eo teste completo de algo-trading podem criar oportunidades rentáveis. (Para mais informações, consulte Como codificar seu próprio robô Algo Trading.)

Jesse Spaulding.
Como fiz $ 500k com aprendizado de máquina e HFT (negociação de alta freqüência)
Esta publicação detalhará o que fiz para fazer aprox. 500k de negociação de alta freqüência de 2009 a 2010. Desde que eu estava negociando completamente de forma independente e não estou mais executando meu programa, eu estou feliz em contar tudo. Minha negociação foi principalmente em contratos de futuros Russel 2000 e DAX.
A chave para o meu sucesso, eu acredito, não estava em uma equação financeira sofisticada, mas sim no projeto de algoritmo geral que uniu muitos componentes simples e a aprendizagem de máquinas usadas para otimizar a máxima rentabilidade. Você ganhou não precisa conhecer qualquer terminologia sofisticada aqui porque, quando eu configurei meu programa, tudo foi baseado na intuição. (O curso de aprendizado de máquina incrível da Andrew Ng não estava ainda disponível - por favor, se você clicar nesse link, você será levado ao meu projeto atual: CourseTalk, um site de revisão para MOOCs)
Primeiro, eu só quero demonstrar que o meu sucesso não foi simplesmente o resultado da sorte. Meu programa fez 1000-4000 negociações por dia (meio e meio, curto) e nunca entrou em posições de mais de alguns contratos por vez. Isso significava que a sorte aleatória de qualquer comércio em particular era muito rápida. O resultado foi que nunca perdi mais de US $ 2000 em um dia e nunca tive um mês perdedor:
(EDITAR: estes números são depois de pagar comissões)
E aqui é um gráfico para dar uma sensação de variação diária. Observe que isso exclui os últimos 7 meses porque - à medida que os números pararam de subir - eu perdi minha motivação para inseri-los.
Antes de configurar meu programa de negociação automatizado I & rsquo; d tinha 2 anos de experiência como um & ldquo; manual & rdquo; comerciante do dia. Isso foi de volta em 2001 - foram os primeiros dias do comércio eletrônico e houve oportunidades para & ldquo; scalpers & rdquo; para ganhar dinheiro. Eu só posso descrever o que eu estava fazendo como semelhante a jogar um jogo de vídeo / jogo com uma suposta vantagem. Ser bem-sucedido significou ser rápido, ser disciplinado e possuir boas habilidades de reconhecimento de padrões intuitivas. Eu consegui fazer cerca de US $ 250 mil, pagar meus empréstimos estudantis e ter dinheiro restante. Ganhar!
Nos próximos cinco anos, eu lançaria duas startups, pegando algumas habilidades de programação ao longo do caminho. Não seria até o final de 2008 que eu voltaria a negociar. Com o dinheiro escorrendo da venda da minha primeira inicialização, a negociação ofereceu esperanças de algum dinheiro rápido enquanto eu descobri minha próxima jogada.
Em 2008 eu estava & ldquo; manualmente & rdquo; dia comercializando futuros usando o software chamado T4. Eu estava desejando algumas teclas de atalho de entrada de pedidos personalizadas, então, depois de descobrir que a T4 tinha uma API, assumi o desafio de aprender C # (a linguagem de programação necessária para usar a API) e segui adiante e me criei algumas teclas rápidas.
Depois de ficar com os pés molhados com a API, logo tive aspirações maiores: queria ensinar o computador a trocar por mim. A API forneceu um fluxo de dados de mercado e uma maneira fácil de enviar ordens para a troca - tudo o que eu tinha que fazer era criar a lógica no meio.
Abaixo está uma captura de tela de uma janela de negociação T4. O que foi legal é que, quando trabalhei, consegui assistir o comércio de computadores nesta mesma interface. Ver as ordens reais que aparecem dentro e fora (por si com meu dinheiro real) foram emocionantes e assustadoras.
O design do meu algoritmo.
Desde o início, meu objetivo era configurar um sistema de forma que eu pudesse estar razoavelmente confiante. Eu ganharei dinheiro antes de fazer qualquer transação ao vivo. Para realizar isso, eu precisava construir uma estrutura de simulação de negociação que, com a maior precisão possível, simulasse a negociação ao vivo.
Embora a negociação no modo ao vivo exigisse o processamento de atualizações de mercado transmitidas através da API, o modo de simulação exigia a leitura de atualizações de mercado a partir de um arquivo de dados. Para coletar esses dados, configurei a primeira versão do meu programa para simplesmente conectar-se à API e registrar as atualizações do mercado com timestamps. Acabei usando 4 semanas de dados de mercado recentes para treinar e testar meu sistema.
Com um quadro básico no local, eu ainda tinha a tarefa de descobrir como criar um sistema comercial lucrativo. Como se verifica, meu algoritmo seria dividido em dois componentes distintos, que eu explorarei por sua vez:
Previsão de movimentos de preços; e fazer negócios lucrativos.
Previsão de movimentos de preços.
Talvez um componente óbvio de qualquer sistema comercial seja capaz de prever onde os preços se moverão. E o meu não foi exceção. Eu definei o preço atual como a média da oferta interna e oferta interna e eu estabeleci o objetivo de prever onde o preço seria nos próximos 10 segundos. Meu algoritmo precisaria apresentar esta previsão momento a momento ao longo do dia de negociação.
Criando & amp; indicadores de otimização.
Eu criei um punhado de indicadores que provaram ter uma habilidade significativa para prever movimentos de preços de curto prazo. Cada indicador produziu um número que era positivo ou negativo. Um indicador era útil se, com maior frequência, um número positivo correspondesse com o mercado subindo e um número negativo correspondia ao mercado descer.
Meu sistema me permitiu determinar rapidamente a capacidade preditiva de qualquer indicador, então eu consegui experimentar muitos indicadores diferentes para ver o que funcionou. Muitos dos indicadores tinham variáveis ​​nas fórmulas que os produziam e consegui encontrar os valores ótimos para essas variáveis, fazendo comparações lado a lado dos resultados obtidos com valores variáveis.
Os indicadores que foram mais úteis foram todos relativamente simples e foram baseados em eventos recentes no mercado que negociei, bem como os mercados de títulos correlacionados.
Fazendo previsões de movimento de preço exato.
Ter indicadores que simplesmente previam um movimento de preços para cima ou para baixo não era suficiente. Eu precisava saber exatamente quanto o movimento do preço era previsto por cada valor possível de cada indicador. Eu precisava de uma fórmula que convertesse um valor indicador para uma previsão de preços.
Para realizar isso, rastreei os movimentos de preços previstos em 50 baldes que dependiam do alcance em que o valor do indicador caiu. Isso produziu previsões únicas para cada balde que eu então consegui representar no Excel. Como você pode ver, a variação esperada do preço aumenta à medida que o valor do indicador aumenta.
Com base em um gráfico como esse, consegui fazer uma fórmula para ajustar a curva. No começo eu fiz isso & ldquo; curve fitting & rdquo; manualmente, mas logo escrevi algum código para automatizar esse processo.
Observe que nem todas as curvas indicadoras tiveram a mesma forma. Observe também que os baldes foram distribuídos logaritticamente de modo a espalhar os dados de forma uniforme. Finalmente, note que os valores de indicadores negativos (e as respectivas previsões de preços descendentes correspondentes) foram invertidos e combinados com os valores positivos. (Meu algoritmo tratado de forma ascendente e exata exatamente o mesmo.)
Combinando indicadores para uma única previsão.
Uma coisa importante a considerar era que cada indicador não era totalmente independente. Eu não poderia simplesmente resumir todas as previsões que cada indicador faz individualmente. A chave era descobrir o valor preditivo adicional que cada indicador tinha além do que já estava previsto. Isso não era muito difícil de implementar, mas isso significava que se eu fosse & ldquo; curve fitting & rdquo; vários indicadores ao mesmo tempo eu tive que ter cuidado; mudar um afetaria as previsões de outro.
A fim de & ldquo; curve fit & rdquo; Todos os indicadores ao mesmo tempo eu configurei o otimizador para passar apenas 30% do caminho para as novas curvas de previsão com cada passagem. Com este salto de 30%, descobri que as curvas de previsão se estabilizariam dentro de algumas passagens.
Com cada indicador agora nos dando a previsão de preço adicional de ñsquo; eu poderia simplesmente adicioná-los para produzir uma previsão única de onde o mercado seria em 10 segundos.
Por que a previsão de preços não é suficiente.
Você pode pensar que com essa vantagem no mercado eu estava dourado. Mas você precisa ter em mente que o mercado é composto por lances e ofertas - não é apenas um preço de mercado. O sucesso na negociação de alta freqüência se resume a obter bons preços e não é tão fácil.
Os seguintes fatores tornam difícil a criação de um sistema lucrativo:
Com cada troca eu tinha que pagar comissões para o meu corretor e a troca. O spread (diferença entre oferta mais alta e oferta mais baixa) significava que, se eu fosse simplesmente comprar e vender aleatoriamente, eu estaria perdendo uma tonelada de dinheiro. A maior parte do volume do mercado eram outros bots que só executariam um comércio comigo se achassem que tinham alguma vantagem estatística. Ver uma oferta não garantiu que eu pudesse comprá-la. No momento em que minha ordem de compra chegou ao intercâmbio, era muito possível que essa oferta tivesse sido cancelada. Como um pequeno jogador do mercado, não havia nenhuma maneira de eu competir sozinho na velocidade.
Construindo uma simulação de negociação completa.
Então eu tive uma estrutura que me permitiu backtest e otimizar indicadores. Mas eu tinha que ir além disso - eu precisava de uma estrutura que me permitisse fazer backtest e otimizar um sistema comercial completo; um onde eu estava mandando ordens e entrando em posições. Neste caso, I & rsquo; d seja otimizado para P & amp; L total e, em certa medida, P & amp; L médio por comércio.
Isso seria mais complicado e, de certa forma, impossível modelar exatamente, mas eu fiz o melhor que pude. Aqui estão algumas das questões que eu tive que lidar com:
Quando um pedido foi enviado ao mercado em simulação, tive que modelar o tempo de atraso. O fato de meu sistema ter visto uma oferta não significava que pudesse comprá-lo imediatamente. O sistema enviaria o pedido, espere aproximadamente 20 milissegundos e, em seguida, apenas se a oferta ainda fosse considerada como um comércio executado. Isso foi inexato porque o tempo de atraso real foi inconsistente e não relatado. Quando eu coloquei lances ou ofertas, tive que olhar para o fluxo de execução comercial (fornecido pela API) e usá-los para avaliar quando minha ordem teria sido executada contra. Para fazer isso, tive que rastrear a posição do meu pedido na fila. (É um sistema de primeira saída em primeiro lugar). Mais uma vez, não consegui fazer isso perfeitamente, mas fiz uma melhor aproximação.
Para refinar a simulação de execução do meu pedido, fiz os meus arquivos de log da negociação ao vivo através da API e comparei-os aos arquivos de log produzidos por negociação simulada do mesmo período. Eu consegui minha simulação até o ponto de ser bastante preciso e, para as partes que eram impossíveis de modelar exatamente, me assegurei pelo menos de produzir resultados estatisticamente similares (nas métricas que achava importantes).
Faz negócios lucrativos.
Com um modelo de simulação de ordem no local, agora eu poderia enviar ordens no modo de simulação e ver uma P & amp; L simulada. Mas como saberia o meu sistema quando e onde comprar e vender?
As previsões de movimento de preços foram um ponto de partida, mas não toda a história. O que eu fiz foi criar um sistema de pontuação para cada um dos 5 níveis de preço na oferta e oferta. Estes incluíram um nível acima da oferta interna (para um pedido de compra) e um nível abaixo da oferta interna (para uma ordem de venda).
Se a pontuação em qualquer nível de preço fosse superior a um certo limite que significaria que meu sistema deveria ter uma oferta / oferta ativa - abaixo do limite, então todas as ordens ativas deveriam ser canceladas. Com base nisso, não era incomum que meu sistema iria mostrar uma oferta no mercado e, em seguida, cancelá-lo imediatamente. (Embora eu tentei minimizar isso, como é irritante, como diabos para quem olha a tela com olhos humanos - inclusive eu.)
Os escores do nível de preços foram calculados com base nos seguintes fatores:
A previsão do movimento do preço (que discutimos anteriormente). O nível de preços em questão. (Os níveis internos significaram que foram necessárias maiores previsões de movimento de preços). O número de contratos na frente do meu pedido na fila. (Menos foi melhor.) O número de contratos por trás do meu pedido na fila. (Mais foi melhor.)
Essencialmente, esses fatores serviram para identificar & ldquo; safe & rdquo; lugares para oferecer / oferecer. A previsão de movimento de preço por si só não era adequada porque não explicava o fato de que ao colocar uma oferta eu não estava preenchido automaticamente - eu só cheguei se alguém me vendesse lá. A realidade era que o simples fato de alguém me vender a um certo preço alterou as probabilidades estatísticas do comércio.
As variáveis ​​utilizadas nesta etapa estavam todas sujeitas a otimização. Isso foi feito exatamente da mesma maneira que otimizei variáveis ​​nos indicadores de movimento de preços, exceto neste caso eu estava otimizando a linha de fundo P & amp; L.
Ao negociar como seres humanos, muitas vezes temos poderosas emoções e desvios que podem levar a decisões menos do que ótimas. Claramente, não queria codificar esses preconceitos. Aqui estão alguns fatores que meu sistema ignorou:
O preço que uma posição foi inserida - Em um escritório de negociação, é muito comum ouvir a conversa sobre o preço no qual alguém é longo ou curto, como se isso pudesse afetar a futura tomada de decisões. Embora isso tenha alguma validade como parte de uma estratégia de redução de risco, ele realmente não tem influência no futuro dos eventos no mercado. Portanto, meu programa ignorou completamente essa informação. É o mesmo conceito que ignorar custos irrecuperáveis. Ir a curto vs. sair de uma posição longa - Normalmente, um comerciante teria critérios diferentes que determinam onde vender uma posição longa versus onde ficar curto. No entanto, da minha perspectiva de algoritmos não havia motivo para fazer uma distinção. Se o meu algoritmo esperava que uma venda de movimento descendente fosse uma boa idéia, independentemente de ser atualmente longa, curta ou plana. A & ldquo; dobrando para cima & rdquo; estratégia - Esta é uma estratégia comum em que os comerciantes comprarão mais ações no caso de o comércio original ir contra elas. Isso resulta em um preço de compra médio menor e significa que quando (ou se) o estoque se virar, você estará configurado para fazer o seu dinheiro de volta em nenhum momento. Na minha opinião, esta é realmente uma estratégia horrível, a menos que você seja o Warren Buffet. Você está enganado para pensar que você está indo bem porque a maioria de seus negócios serão vencedores. O problema é quando você perde você perder grande. O outro efeito é que dificilmente julgar se você realmente tem uma vantagem no mercado ou está apenas tendo sorte. Ser capaz de monitorar e confirmar que o meu programa de fato teve uma vantagem foi um objetivo importante.
Uma vez que meu algoritmo tomou decisões do mesmo modo, independentemente de onde ele entrou em um comércio ou se fosse atualmente longo ou curto, ocasionalmente sentava-se (e aceitou) alguns grandes negócios perdidos (além de alguns grandes negócios vencedores). Mas, você não deveria pensar que não havia nenhum gerenciamento de riscos.
Para gerenciar o risco, apliquei um tamanho máximo de posição de 2 contratos por vez, ocasionalmente acumulado em dias de alto volume. Eu também tive um limite máximo de perda diária para proteger contra quaisquer condições de mercado inesperadas ou um erro no meu software. Esses limites foram aplicados no meu código, mas também no backend através do meu corretor. Como aconteceu, nunca encontrei problemas significativos.
Desde o momento em que comecei a trabalhar no meu programa, demorei cerca de 6 meses antes de chegar ao ponto de rentabilidade e começar a executá-lo ao vivo. Embora seja justo, uma quantidade significativa de tempo foi aprender uma nova linguagem de programação. Enquanto trabalhava para melhorar o programa, vi maiores lucros para cada um dos próximos quatro meses.
Todas as semanas, eu treinaria o sistema com base nas 4 semanas anteriores de dados. Eu achei que isso atingiu o equilíbrio certo entre a captura de tendências comportamentais recentes do mercado e garantir que meu algoritmo tivesse dados suficientes para estabelecer padrões significativos. À medida que o treinamento começou a tomar mais e mais tempo, eu o separei para que ele possa ser executado por 8 máquinas virtuais usando o Amazon EC2. Os resultados foram então agrupados na minha máquina local.
O ponto alto da minha negociação foi outubro de 2009, quando eu fiz quase 100k. Depois disso, continuei a gastar os próximos quatro meses tentando melhorar meu programa, apesar da diminuição do lucro a cada mês. Infelizmente, neste ponto, acho que eu implementei todas as minhas melhores idéias, porque nada que tentei pareceu ajudar muito.
Com a frustração de não poder fazer melhorias e não ter um senso de crescimento, comecei a pensar em uma nova direção. Eu enviei 6 empresas de comércio de alta freqüência diferentes para ver se eles estavam interessados ​​em comprar meu software e me contratar para trabalhar para eles. Ninguém respondeu. Eu tive algumas idéias de inicialização novas que queria trabalhar, então eu nunca segui.
UPDATE - Posteci isso no Hacker News e tem tido muita atenção. Eu só quero dizer que não defendo ninguém tentando fazer algo assim agora. Você precisaria de uma equipe de pessoas realmente inteligentes com uma variedade de experiências para ter alguma esperança de competir. Mesmo quando eu estava fazendo isso, eu acreditava que era muito raro que os indivíduos conseguissem sucesso (embora eu tivesse ouvido falar de outros).
Há um comentário no topo da página que menciona "estatísticas manipuladas" e se refere a mim como um investidor de varejo & ldquo; rdquo; que os quants gostariam de escolher com entusiasmo & rdquo ;. Este é um comentário bastante infeliz que simplesmente não é baseado na realidade. Configurando isso de lado há alguns comentários interessantes: news. ycombinator / item? Id = 4748624.
UPDATE # 2 - I & rsquo; postou um FAQ de seguimento que responde algumas perguntas comuns que eu recebi dos comerciantes sobre esta publicação.
Delhideviant gostou disto.
Oi, sou Jesse, fundador da Thinklab. Eu vivo e toco em São Francisco. Você encontrou minha casa na web ... Bem-vindo!

The Idiots Guide to High Frequency Trading.
Primeiro, deixe-me dizer que o que lê aqui vai estar errado de várias maneiras. A HFT abrange um caminho tão abrangente de negociação que diferentes partes participam ou são afetados de diferentes maneiras. Eu queria colocar isso lá fora como ponto de partida. Esperemos que os comentários ajudem a educar-nos a todos.
1. A negociação eletrônica faz parte da HFT, mas nem todas as negociações eletrônicas são operações de alta freqüência.
As ações de negociação e outros instrumentos financeiros existem há muito tempo. É a Negociação Eletrônica que levou a spreads muito menores e custos de negociação reais mais baixos do seu corretor. Muitas vezes, as empresas HFT acreditam pela redução de spreads. Eles não. O comércio eletrônico fez.
Todos nós negociamos eletronicamente agora. Não é grande coisa.
As pessoas gostam de ver a velocidade da negociação como o problema. Não é. Tivemos uma necessidade de velocidade desde que as primeiras cotações de ações foram comunicadas através do país via telégrafo. A busca de velocidade nunca foi encerrada. Embora eu não pense que a localização e a sub-segunda negociação agregam valor ao mercado, NÃO cria problemas para o mercado.
3. Sempre houve um delta na velocidade de negociação.
Desde os dias do telegrama acima mencionado até o comércio sub milissegundo, nem todos trocaram à mesma velocidade. Você pode negociar ações em uma conexão de banda larga de 100mbs que é mais rápida do que a conexão de discagem de seus vizinhos. Esse delta em velocidade dá-lhe informações mais rápidas para notícias, informações, pesquisas, obter cotações e obter seus negócios para seu corretor mais rápido.
O mesmo se aplica aos corretores, bancos e HFT. Eles competem para obter a velocidade mais rápida possível. Novamente, a velocidade não é um problema.
4. Então, o que mudou? Qual é o problema.
O que mudou é esse. No passado, as pessoas usavam suas vantagens de velocidade para negociar suas próprias carteiras. Eles sabiam que tinham uma vantagem com informações mais rápidas ou colocação de negócios e eles o usavam para comprar e possuir ações. Se for por horas. Isso é aceitável. O mercado é muito darwinista. Se você pudesse descobrir como aproveitar a velocidade para comprar e vender ações que você adquiriu, mais poder para você. Se você trocou o dia em 1999, porque você conseguiu ver o movimento em ações mais rápido do que o cara em discagem, e você ganhou dinheiro. Mais poder para você.
O que mudou é que os intercâmbios entregavam informações mais rapidamente para aqueles que pagavam o direito E TAMBÉM lhes deram a habilidade através de tipos de pedidos onde os comerciantes mais rápidos tinham o direito de saltar na frente de todos aqueles que eram mais lentos (Traders se sentem livres para desafiar Eu neste assunto). Não só isso, eles foram capazes de usar algoritmos para ver a atividade e / ou ver diretamente as cotações de todos aqueles que foram mesmo milissegundos mais lentos.
Com essas mudanças, os jogadores mais rápidos agora podiam ganhar dinheiro simplesmente porque eram os comerciantes mais rápidos. Eles não se importaram com o que eles trocaram. Eles perceberam que poderiam ganhar dinheiro com o chamado Arbitragem de Latência. Você ganha dinheiro sendo o mais rápido e aproveitando os comerciantes mais lentos.
Não importava quais trocas os negócios estavam, ou se fossem entre trocas. Se fossem mais rápidos e pudessem ver ou antecipar os negócios mais lentos, eles poderiam lucrar com isso.
É aí que os problemas começam.
Se você tem o acesso mais rápido à informação e as trocas lhe deram incentivos para saltar na frente desses usuários e fazer negócios, pagando você por qualquer volume que você crie (fabricante / comprador), então você pode usar essa combinação para fazer negócios que você são praticamente GARANTIDOS PARA FAZER UM LUCRO.
Então, basicamente, os jogadores mais rápidos, que gastaram bilhões de dólares em conjunto para obter o acesso mais rápido possível, estão usando essa velocidade para saltar para a frente da linha de negociação. Eles conseguem ver, diretamente ou algorítmicamente, os negócios que estão entrando no mercado.
Quando digo algorítmicamente, isso significa que as empresas estão usando sua velocidade e sua força intelectual para tomar tantos pontos de dados como podem usar para prever o que os negócios irão acontecer a seguir. Isso não é fácil de fazer. É muito difícil. É preciso pessoas muito inteligentes. Se você criar algoritmos vencedores que podem antecipar / prever o que acontecerá nos próximos milissegundos em mercados / ações, você ganhará milhões de dólares por ano. (Nota: nem todos os algoritmos são ruins. Algoritmos são apenas funções. O que importa é a intenção deles e como eles são usados)
Esses algoritmos tomam qualquer número de pontos de dados para direcionar onde e o que comprar e vender e eles fazem o mais rápido possível. A velocidade de processamento também é um problema. Até o ponto em que existem CPUs especiais usadas para processar conjuntos de instruções. Em termos simples, o mais rápido possível, se pensarmos que isso acontecerá, então faça isso.
A saída dos algoritmos, The This Then That cria o comércio (novamente esta é uma simplificação, estou aberta a melhores exemplos), o que cria um lucro de uma quantidade relativamente pequena. Quando você faz isso milhões de vezes por dia, isso totaliza dinheiro real. IMHO, esta é a definição de negociação de alta freqüência. Aproveitando uma vantagem em velocidade e processamento algorítmico para saltar na frente de negociações de participantes de mercado mais lentos para criar pequenas vitórias garantidas milhões de vezes ao dia. É necessária uma alta freqüência de negócios para ganhar dinheiro.
Ai que está o problema. Este é o lugar onde o jogo é manipulado.
Se você sabe que, ao chegar à frente da linha, você pode ver ou antecipar algum número material dos negócios que estão prestes a acontecer, você está GARANTIDO para obter lucro. Qual é a definição de um mercado fraudado? Quando você é garantido para obter um lucro. Em termos de cassino, o comerciante que possui a frente da linha é a casa. A casa sempre ganha.
Então, quando Michael Lewis e outros falam sobre o mercado de ações sendo manipulados, é disso que eles estão falando. Você pode dizer que o mercado de ações INTEGRAL é manipulado, mas você pode dizer que, para as ações / índices onde HFT joga, o jogo é manipulado para que os jogadores mais rápidos e inteligentes garantam o dinheiro.
6. Isso é ruim para os investidores individuais?
Se você comprar e vender ações, por que você se importaria se alguém aproveite seu investimento em velocidade para fazer alguns centavos de você? Você decide, mas aqui está o que precisa saber:
uma. Milhões de dólares foram gastos para chegar à frente da linha. Todos esses comerciantes que investiram em velocidade e algoritros de algoritmos caros precisam obter um retorno sobre seu investimento. Eles fazem isso saltando na frente do seu comércio e scalping apenas um pouco. O que aconteceria se eles estivessem lá? Há uma boa chance de que qualquer lucro que eles tenham feito saltando na frente do seu comércio iria para você ou seu corretor / banqueiro.
b. Se você trocar pequenas ações, isso não afetará pequenas ações. A HFT não lidou com estoques de baixo volume. Por definição, eles precisam fazer uma alta freqüência de negócios. Se os estoques que você comprar ou vender não tem volume (eu não sei qual é o volume mínimo de volume), então eles não estão mexendo com seus estoques.
c. Isso é um problema de ética para você e outros investidores? Se você acredita que os investidores se afastarão do mercado porque sentem que é errado éticamente para qualquer parte do mercado oferecer aos poucos selecionados uma maneira garantida de ganhar dinheiro, então poderia gerar fluxos significativos de investidores em dinheiro que poderiam impacto no seu patrimônio líquido. IMHO, é por isso que Schwab e outros corretores que lidam com investidores de varejo estão preocupados. Podem perder clientes que pensam que Schwab, etc. não podem acompanhar outros corretores ou não estão encaminhando suas ordens tão eficientemente quanto outros.
7. Existem riscos sistêmicos que resultam de tudo isso.
A resposta simples é que, pessoalmente, acredito que, sem dúvida, a resposta é SIM. Por quê ?
Se você sabe que um jogo é manipulado e que é LEGAL para participar deste jogo fraudado, você faria todo o possível para participar, se pudesse?
Claro que sim. Mas isso não é um novo fenômeno. A batalha para capturar todo esse dinheiro garantido está acontecendo há vários anos. E o que aconteceu é muito darwinista. Os jogadores mais inteligentes subiram ao topo. Eles estão capturando grande parte da pilhagem. Realmente é uma corrida de armamentos. Mais velocidade dá-lhe mais slots na frente das linhas. Então, mais dinheiro está sendo gasto em velocidade.
O dinheiro também está sendo gasto em algoritmos. Você precisa do melhor e mais brilhante para escrever algoritmos que lhe fazem dinheiro. Você também precisa saber como influenciar os mercados para dar aos seus algoritmos a melhor chance de ter sucesso. Existe um problema nos mercados conhecidos como recheio de cotações. É aí que o HFT cria citações que supostamente enganam outros algoritmos, comerciantes e investidores para acreditar na sua verdade é uma verdadeira ordem disponível para ser atingida. Na realidade, essas não são ordens reais. Eles são chamarizes. Em vez de permitir que alguém acerte a ordem, porque eles são mais rápidos do que todos os outros, eles podem ver sua intenção de atingir a ordem ou sua reação diretamente ou algorítmica ao orçamento e agir. E não só isso, cria um volume tão grande de fluxo de informações que torna mais caro para todos os outros processarem essas informações, o que, por sua vez, as desacelera e as coloca mais em desvantagem.
IMHO, isso não é justo. Não é uma intenção real. No coração é uma FRAUDE NO MERCADO. Nunca houve a intenção de executar um comércio. É apenas para enganar.
Mas Order Stuffing não é o único problema.
Todo mundo no negócio HFT quer chegar à frente da linha. Eles querem esse dinheiro garantido. Para chegar lá, o HFT não só usa velocidade, mas eles usam algoritmos e outras ferramentas (sinta-se livre para fornecer mais informações aqui pessoal HFT) para tentar influenciar outros algoritmos. É preciso uma certa quantidade de arrogância para ser bom no HFT. Se você acha que pode pensar em outras empresas da HFT, você vai tentar enganá-las a tomar ações que causem que seus algoritmos não troquem ou façam negócios ruins. É análogo aos grandes jogadores de poker contra o resto de nós.
O que nós não sabemos é exatamente o quão longe as empresas HFT e seus algoritmos irão chegar à frente da linha. Há um risco moral envolvido. Será que eles correrão riscos sabendo que se eles falharem, eles podem perder seu dinheiro, mas os resultados também podem ter implicações sistêmicas? Vimos o que aconteceu com o Flash Crash. Existe alguma maneira de evitar que o mesmo aconteça de novo? Eu não penso assim. É possível que algo bem pior possa acontecer? Eu não faço ideia. E ninguém mais.
É essa falta de capacidade de quantificar os riscos que geram um enorme custo para todos nós. Warren Buffet chamou armas derivadas de destruição em massa porque ele teve e não tem idéia do potencial impacto negativo de um ator ruim. O mesmo problema se aplica ao HFT. Como pagamos por esse risco? E quando ?
Quando você tem algoritmos HFT lutando para chegar à frente da linha para obter esse dinheiro garantido, quem sabe até que ponto eles correrão riscos e o que eles impactarão não será apenas nos nossos mercados de ações dos EUA, mas também moedas, mercados estrangeiros e ? ? ?
E o que os jogadores da HFT estão fazendo agora fora dos mercados dos EUA? Todos os mercados estão correlacionados em algum nível. Problemas fora dos EUA podem criar enormes problemas para nós aqui.
IMHO, existem problemas sistêmicos reais em jogo.
8. Então, por que alguns dos grandes bancos e fundos não estão gritando um assassinato sangrento?
Para usar uma analogia de black jack, é porque eles sabem como contar cartões. Eles têm os recursos para descobrir como combinar as empresas de HFT mais rápidas em suas velocidades de negociação. Eles podem dar ao luxo de comprar a velocidade ou podem se associar com aqueles que podem. Eles também têm o poder intelectual para descobrir genericamente como os algoritmos funcionam e onde eles estão scalping seus lucros. Ao saber disso, eles podem evitá-lo. E porque eles têm o poder do cérebro para descobrir isso, eles podem realmente usar o HFT para sua vantagem de tempos em tempos. Onde eles podem ver o HFT no trabalho, eles podem alimentá-los com trades que proporcionam alguma liquidez real em oposição ao volume.
O ponto seguinte, é claro, é que, se os grandes podem fazer isso, e os pequenos podem deixar que os grandes conseguissem o dinheiro deles, não devemos simplesmente nos calar e trabalhar com eles? Claro que não. Não devemos investir apenas com as maiores empresas para evitar alguns dos riscos da HFT. Devemos ser capazes de tomar nossas decisões como investidores para trabalhar com aqueles que nos dão o melhor suporte para fazer investimentos. Não são aqueles que têm a melhor solução para superar HFT.
Mas o que é mais importante, mesmo o maior e mais inteligente dos comerciantes, aqueles que podem ver e antecipar as ações das empresas HFT não podem explicar as ações de atores ruins. Eles podem acompanhar a corrida de armamentos para chegar à frente da linha. Não é a sua competência central. É um problema para eles, mas eles também sabem que ao poder lidar com isso melhor do que seus pares, isso lhes dá uma vantagem de venda. & # 8220; Nós podemos lidar com HFT sem problema & # 8221 ;. Então eles não estão gritando assassinato sangrento.
IMHO, não vale o risco. Eu sei por que há HFT. Eu simplesmente não vejo por que deixamos isso continuar. Não adiciona valor. Mas se isso continuar, então devemos exigir que todos os jogadores ALGORITMIC para registrar seus Algoritmos. Enquanto eu não sou fã da SEC, eles têm jogadores inteligentes em seu grupo de estrutura de mercado. (o valor de ir para a SEC Speaks :). Embora as cópias dos algoritmos bloqueados na SEC não impeçam um colapso / colapso no mercado, pelo menos, podemos reverter o engenheiro se acontecer.
Eu sei que isso parece estúpido no rosto. O engenharia reversa de um colapso? Mas essa pode ser uma solução melhor do que esperar que a SEC descobre como regular e evitar um crash do mercado.
10 & # 8230; PENSAMENTOS FINAIS FINAIS.
Eu escrevi isso em cerca de 2 horas. Não porque achei que seria definitivo ou correto. Espero obter ABSOLUTAMENTE CRUSHED em muitos pontos aqui. Mas há tão pouco conhecimento e compreensão do que está acontecendo com a HFT, que eu acreditava que alguém precisava iniciar a conversa.
Diga ao mundo por favor!
134 pensamentos sobre & ldquo; The Idiots Guide to High Frequency Trading & rdquo;
bela postagem. obrigado.
I noticed that many respondents, come up with frightening scenarios about imminent apocalyptic crashes precipitated by HFT malfeasance, dwarfing the Great Recession and “Flash crash”.
They would be great screenplays for “Armageddon”-like movies, but I am sorry to inform you, that for many reasons they cannot happen any more, so tonight you can sleep little better.
Most, if not all exchanges impose various breaks and limits on HFT transactions as follows:
* Number of messages/sec: (ex. CME – 1,000 msgs/sec), all messages exceeding the limit are discarded by the exchange, if you reach the limit in 0.1 sec or 1 millisecond, for the remaining 0.9 sec or 0.999 sec all your messages will be discarded by the exchange, and you will be for all practical purposes disconnected from the exchange unable to trade.
In addition, if after reaching the limit, you continue to flood the exchange with messages during the remaining period when you should be idle, if it persists for a while, you will be unceremoniously disconnected from the exchange network and questioned by friendly folks from the exchange compliance office trying to find out what happened, was it an unintentional software or hardware error or was it more intentional?
If this scenario happens frequently, you will be simply banned from trading on that exchange.
That takes care of the denial of service type of abuse and frequently repeated fairy-tale scenario of HFT firms executing millions of trades per second.
* Ratio of messages to trades: (ex. CME – 500 msgs/trade) forcing traders to trade, fines for exceeding the limit.
It limits so-called quote-stuffing or spoofing, whereby you would send thousands of “fake” orders to pull the market in one direction and then suddenly cancel all these orders and reap ill-gotten rewards on the rebound.
Regulators are also looking for that type of trading behavior, and if they find it, there are very bitter consequences.
* Circuit breakers limiting price movement: very simple, the exchange computers monitor price movements for all traded instruments. If a particular instrument price changes by more than a specified percentage during specified time interval, the trading in that instrumented is simply suspended for a specified “cooling off” period, after which it is carefully restarted.
For example + or – 10% price change in 30 min, trading is suspended.
These limits assure that another “Flash crash” or worse is simply not possible to occur again, anyway not one caused by HFT.
If the economy tanks or we have another Credit Default Swap precipitated bubble followed by losses of $8 trillion, then all bets are off, but please don’t blame it on HFT (by the way, Flash crash was not caused by HFT trading but by a “fat finger”, and its severity was exacerbated by the lack of Circuit breakers implemented as the result of it).
Finally, I’d like to offer some interesting statistics.
In 2012 entire HFT profits in US were $1.25bn and according to WSJ total 2013 HFT investment was $1.5bn.
All that for facilitating estimated 50+% of all financial transactions worth tens of trillions of dollars.
These numbers don’t even register as rounding errors as compared with other financial sectors or even individual companies, yet the public is in frenzy calling for burning HFT at the stake, fueled, by among others, the recent 60 Minutes program and book by Michael Lewis, both full of factual errors and conspiracy theories, as well as recent subpoenas of HFT trading companies by NYAG Mr. Schneiderman, who no doubt would like to get the recognition like his predecessors Mr. Spitzer and Major Giuliani.
Wouldn’t you expect HFT to show enormous profits, at least matching the level of alleged abuses and world-wide hysteria, instead of paltry $1.25bn?
I’ll let you draw your own conclusions.
Ultra-low Latency Architect/Developer.
The market is Darwinian, and always will be. The same applies to algorithms that govern search engine rankings. Except Google has a monopoly, and they dictate the rules of the game.
Mr. Cuban, I have a reality show competition concept that I believe you will be interested in. I won’t waste your time by hyping it up on your blog. If you are willing to refer me to an agency that will represent me I will give you the first right of refusal to purchase an interest in this concept. I appreciate any assistance you are willing to provide me. Sincerely, Justin Weber.
As one of the idiots on this topic, I hasten to offer a BIG “thank you” to Mark. His post is extremely helpful to me, and it has triggered a fabulous discussion during which a number of experts more or less confirmed most of his central points. Please allow this idiot to raise some questions/observations from the “peanut gallery”, addressing algo usage in general.
1. Wouldn’t the algorithms tend to produce the same trading response, subject to some minor random variation, given the same set of values on their input variable’s? The classes of variables that are relevant and their pertinent ranges of variation are not terribly difficult to imagine. Also, among these variables the values often tend to make “recurrent tracks” across their ranges, rather than to continually move into brand new territory (earnings per share reports, e. g.) . This leads me to wonder whether “algo behavior” tends to set up certain recurrent price change patterns, certain kinds of short-term waves of price movement, for example. If so, might it not be profitable for individual traders to find out how to sense these patterns and capitalize upon them in their trading? For example, if my memory is correct going back to the 80s and 90s we might in those days have to wait weeks for a popular stock to move to certain ” deemed to be extreme” price levels and set up special buying or selling opportunity, whereas now we may have to wait only a few days or even one day to see such stocks move to deemed extremes (or by extreme amounts from heir initial positions).
2. If we view a market crash as the result of contagion of fear within a crowd, and during the “reign of fear” the market might fall X% over a number of days, should we not expect that in future crashes it may well take minutes, rather than days, for a fall of X% to take place? (n other words, the Flash Crash may have been an isolated event; but the now prevailing technologies (widespread use of algos) imply that under the right set of fear triggers crashes of similar or greater magnitude should be expected. While algos do not feel fear, their software logics might often be such that a network of algos interacting could easily simulate intense fear contagion (self-reinforcing selling waves or massive withdrawal of bids) . Is this all nonsense?
If not, the implications for the individual investor are enormous, I think. For example, we must intensify our usage of insurance positions via options.
3. While I see that my trades are most likely handled internally by the broker, I feel uneasy about the apparent fact that time honoured principles and procedures in interpreting published price patterns may be becoming obsolete as a result of algo behaviour.
IMHO there are few points that are being glossed over in the recent HFT discussion.
1. HFT supporters can often be heard taking credit for an increase in liquidity. Which I believe is really more of an increase in volume as opposed to real liquidity, but even if I concede that true liquidity has increased, I wonder, was it really needed? Has traditional electronic trading not created a sufficient opportunity to efficiently match real buyers and sellers? Idk, but perhaps it’s worth discussing,
2. While one, two, or ten HFT firms and their latency arbitrage strategies existing in the market may have a negligible effect on market orders I wonder if we have reached a toxic level of participation. Michael Lewis mentioned that over 99 percent of all open orders are sent in by HFT firms and that roughly 50% of all filled orders can be attributed to HFT. Now I have no idea how accurate those stats truly are, but if they are even in the ballpark then I think we, as market participants, should be concerned about an overdose.
3. Mark mentions systemic risk to the global marketplace and I believe that therein lies the rub with regulation. It’s impossible to determine if our current market environment is being propped up by millions of arbitrage capitalists or not. If the last five years of “growth” has been synthetic, to what degree will the global markets be affected if our regulators remove HFT firms and their ilk from the exchanges? I think the uncertainty around the answer is one reason we see such a subdued reaction across the board.
Obviously there are several sides to this conversation and I’m simply following Mark’s lead, hoping to extend the discourse. Deixe-me saber a sua opinião.
I have lot of qualitative posts at your blog, and this being one of them.
For those that may not have the time to pick up Michael Lewis’ new book entitled “Flash Boys: A Wall Street Revolt” here is a fairly good high level introduction to high frequency trading by Mark Cuban. I also suggest reading Michael Lewis’ article in the NY Times entitled “The Wolf Hunters of Wall Street” nyti. ms/1iRYw3Y. It is a major issue that regulators have to tackle to restore confidence in the financial markets.
edpalermo IMHO the way to counter this is through technology. It seems that IEX has found the answer to the speed advantage. Other exchanges can also follow their lead. I just don’t see how you police this properly, it is a cutting edge technology race. Can you penalize the most efficient without taking away incentives to improve efficiencies?
I know this is not the right blog to be posting this on. I also know that this will probably never be responeded to, but I don’t want to go through life wandering “what if”. I am a teacher that has just recently been let go due to the dwindling number of students at our school. I have a child on the way and am sitting on an idea that I have kept stored away until I had the courage to persue it. The idea will make a very large profit for anybody that invests in the initial start-up. Mark, if you do happen to stumble upon this, I would be more than happy to discuss the details with you. I have already been doing some research, and have not found anything like what I have to offer. This idea doesn’t focus on a specific demographic and will help people in HUGE ways across the USA. All I need is a partner that can help me get the whole thing up and running. I do have some people that are interested in the idea and are wanting to do lunch next week to discuss how to get it started, but I wanted to partner up with somebody that has experience in this field. Hope all is well MC! GO MAVS!
Mark, good writeup to start the conversation. I largely agree, except the small guy is penalized. Most “small guys” have their money invested in the market via mutual funds. These vehicles process enormous bulk trades since they are representing a pool of assets. So although the small guy might not get whacked trading individual stocks… since the bulk of their wealth is in mutual funds and 401k plans, they are actually the ones that suffer from HFTs frontrunning their trades.
The stock market is structured in a way that allows the big guy to fleece the little guy. HFT is just another chapter in that story. I think it would be great if we could create a format and the regulations that would make the market more “fair” and give individual investors a fighting chance. But that seems really unlikely, due to the huge amounts of potential profit being made under the current system. And the fact that no matter what new laws you put in place, there will always be really smart and well financed people with a lot to gain by figuring out the loopholes. If you think of this as two competing sides, doesn’t the side with an unfair advantage have a lot more incentive (and financial muscle) than the side represented by the SEC and the individual investor?
Great food for thought from streetwise professor.
“Pinging: Who is the Predator, and Who Is the Prey?”
The debate over Lewis’s Flash Boys is generating more informed commentary than the book itself. One thing that is emerging in the debate is the identity of the main contending parties: HFT vs. the Buy Side, mainly big institutional traders.
One of the criticisms of HFT is that it engages in various strategies to attempt to ferret out institutional order flows, which upsets the buy side. But the issue is not nearly so clearcut as the buy side would have you believe.
The main issue is that not all institutional orders are alike. In particular, there is considerable variation in the informativeness of institutional order flow. Some (e. g., index fund order flow) is unlikely to be informed. Other order flow is more informed: some may even be informed by inside information.
Informed order flow is toxic for market makers. They lose on average when trading against it. So they try to determine what order flow is informed, and what order flow isn’t.
Informed order flow must hide in order to profit on its information. Informed order flow uses various strategies based on order types, order submission strategies, choice of trading venues, etc., to attempt to become indistinguishable from uninformed order flow. Uninformed order flow tries to devise in strategies to signal that it is indeed uninformed, but that encourages the informed traders to alter their strategies to mimic the uninformed.
To the extent that market makers-be they humans or machines-can get signals about the informativeness of order flow, and in particular about undisclosed flow that may be hitting the market soon, they can adjust their quotes accordingly and mitigate adverse selection problems. The ability to adjust quotes quickly in response to information about pending informed orders allows them to quote narrower markets. By pinging dark pools or engage in other strategies that allow them to make inferences about latent informed order flow, HFT can enhance liquidity.
Informed traders of course are furious at this. They hate being sniffed out and seeing prices change before their latent orders are executed. They excoriate “junk liquidity”-quotes that disappear before they can execute. Because the mitigation of adverse selection reduces the profits they generate from their information.
It can be frustrating for uninformed institutional investors too, because to the extent that HFT can’t distinguish perfectly between uninformed and informed order flow, the uninformed will often see prices move against them before they trade too. This creates a commercial opportunity for new trading venues, dark pools, mainly, to devise ways to do a better way of screening out informed order flow.
But even if uninformed order flow often finds quotes running away from them, their trading costs will be lower on average the better that market makers, including HFT, are able to detect more accurately impending informed orders. Pooling equilibria hurt the uninformed: separating equilibria help them. The opposite is true of informed traders. Market makers that can evaluate more accurately the informativeness of order flow induce more separation and less pooling.
Ultimately, then, the driver of this dynamic is the informed traders. They may well be the true predators, and the uninformed (or lesser informed) and the market makers are their prey. The prey attempt to take measures to protect themselves, and ironically are often condemned for it: informed traders’ anger at market makers that anticipate their orders is no different that the anger of a cat that sees the mouse flee before it can pounce. The criticisms of both dark pools and HFT (and particularly HFT strategies that attempt to uncover information about trading interest and impending order flow) are prominent examples.
The welfare impacts of all this are unknown, and likely unknowable. To the extent that HFT or dark pools reduce the returns to informed trading, there will be less investment in the collection of private information. Prices will be less informative, but trading will be less costly and risk allocation improved. The latter effects are beneficial, but hard to quantify. The benefits of more informative prices are impossible to quantify, and the social benefits of more informed prices may be larger, perhaps substantially so, than the private benefits, meaning that excessive resources are devoted to gathering private information.
More informative prices can improve the allocation of capital. But not all improvements in price efficiency improve the allocation of capital by anything near the cost of acquiring the information that results in these improvements, or the costs imposed on uninformed traders due to adverse selection. For instance, developing information that permits a better forecast of a company’s next earnings report may have very little effect on the investment decisions of that company, or any other company. The company has the information already, and other companies for which this information may be valuable (e. g., firms in the same industry, competitors) are going to get it well within their normal decision making cycle. In this case, incurring costs to acquire the information is a pure waste. No decision is improved, risk allocation is impaired (because those trading for risk allocation reasons bear higher costs), and resources are consumed.
In other words, it is impossible to know how the social benefits of private information about securities values relate to the private benefits. It is quite possible (and in my view, likely) that the private benefits exceed the social benefits. If so, traders who are able to uncover and anticipate informed trading and take measures that reduce the private returns to informed trading are enhancing welfare, even if prices are less informative as a result.
I cannot see any way of evaluating the welfare effects of financial trading, and in particular informed trading. The social benefits (how do more informative prices improve the allocation of real resources) are impossible to quantify: they are often difficult even to identify, except in the most general way (“capital allocation is improved”). Unlike the trade for most goods and services, there is no reason to believe that social and private benefits align. My intuition-and it is no more than that-is that the bulk of informed trading is rent seeking, and a tax on the risk allocation functions of financial markets.
It is therefore at least strongly arguable that the development of trading technologies that reduce the returns to informed trading are a good thing. To the extent that one of the charges against HFT-that it is better able to detect and anticipate (I will not say front-run) informed order flow-is true, that is a feature, not a bug.
I don’t know and I am pretty sure nobody knows or even can know the answers to these questions. Which means that strongly moralistic treatments of HFT or any other financial market technology or structure that affects the returns to informed trading is theology, not economics/finance. Agnosticism is a defensible position. Certitude is not.
Mr. Cuban if you can spare the time please help a blue collar guy understand ?
Do stock markets benefit to society?
(is this a place where need for capital is matched with investors?)
Does HFT provide any benefit to those wishing to raise capital?
Does HFT provide any benefit to investors?
Does HFT provide any benefit to anyone other than the owner of the HFT system?
Have I read comments by yourself to the effect that taxing trades not held for a certain time period would discourage this practice?
Would not the money absorbed by HFT not be better (for society as a whole) invested in building businesses instead?
The problem is not technology simpliciter.
What frustrates me about this whole discussion is the fact that a lot of incorrect things are being said and certain facts are slightly adjusted where they become juicy fiction all to fit better in the fairy tale of the evil HFT firms rigging the market in conspiracy with the exchanges. Are all HFT honest participants with a fair market in their mind? No. Should they be closely monitored and should regulations adapt to their developments to ensure the market stays free and fair and stable? Absolutamente. There are certainly risks in HFT that could affect the market (and economy!) as a whole (find me an HFT firm who denies this!). But a wise professor once told me “You should be against something for the RIGHT reasons”:
First it’s obvious that jealousy should be avoided in a rational discussion of HFT. If you think the only thing wrong with HFT firms is that they make too much money, you should try to tax them more or change the rules of the markets to prevent them from making as much profit as they do (transaction tax?). Or perhaps you should start a HFT firm yourself, there are plenty of people who have enough money to start one, if successful the dividends tend to be high so investors should not be hard to find if your plans are sound. Hint: the biggest challenge is not having the money, but having the expertise and the human capital.
The Alleged front-running: Front running in the strict sense of the word is highly illegal. If a broker gets a limit order from a customer, which she is free to manage on his behalf, it means that the broker is 100% sure her customer will buy/sell at that volume/price as that is what an order is (the probability of a customer not honouring his order is negligible). If the broker finds a better price, OTC or on the exchange, she should give that better price to her customer and only receive the agreed brokerage fee. If a broker buys/sells the product herself to then make money at the expense of her customer, she is front running and ripping of her customer and this is obviously illegal.
The term front running in this discussion of HFT is abused. A HFT firm’s algo’s that trade on the public exchanges only have access to the public information available to everyone else and are only able to make trading decisions based on this public feed (nevertheless within 300 microseconds). Orders on exchanges can be cancelled at any time (again within 300 microseconds). There is no way to know the origin of an order on the exchange, in fact the only thing that is visible is the order book (how much volume at which price level). When the volume at a price level changes from 100 to 105, it is very likely that someone sent in an order of volume 5 at that level, but there is no way to know whether this order is from any of the same persons that sent in the orders making up the initial 100 volume… The exchange operate at FIFO principle (first in, first out). No matter the origin of the order, if it arrives first at the exchange it will be the first one to be filled by a matching opposite order (buy vs sell). It is impossible to jump in front of an order once it has arrived at the exchange’s matching engine.
In what way then can there be a perception/fear of front running by HFT? Due to the speed advantage and processing advantage (faster computers), and intelligence advantage (fast and clever algo’s), HFT firms are faster than other market participators to spot a movement of the market (due to ask and demand or otherwise) by interpreting the public feed. Say a HFT market maker has the following orders in the market: buy: 10099 50100 — sell: 50102 100103. Now a Pension fund sent in a large order to buy 500 stocks via a brokerage firm. The brokerage firm decides to put in an iceberg order to not cause the market to move against her (if she would just put in the market order, other market participants will see her eagerness to buy and think ‘oh lets see how badly she wants it, let me higher my prices’). So she puts in an iceberg order slicing up the original order in slices of 50 volume (only putting in orders of 50 volume after each other when the previous slice is filled). after the first 50 volume order at 102 of the HFT firm is filled, the algorithm automatically fills up the level again: 10099 50100 — 50102 100103. The iceberg order will trigger again another trade at 50102. Now the HFT firm has a short position of 100 and just did 2 consecutive sells. This means there is an increase in demand, while his position is short! The HFT firm needs to retreat or risks losing money… 100100 50101 — 50103 100104. the algo of the HFT firm adjusts its prices upward for two reasons: 1. It wants to make it more attractive for other participants to sell to him (e. g. 2×50101, and make 1 dollar/share on buying back the 100 shares sold at 102). 2. ask more credit for participants that are willing to sell (at 103 instead of 102). When the algo anticipates very big movements, it might even decide to move prices up quite a lot. By moving up prices to 100101 50102 — 50104 100105 the HTF firm will very likely buy the stocks at 102 from other participants who were not fast enough to move their prices (because they are unaware of the buying pressure or their computers were to slow to react), before the big order is completely filled it will have moved prices up past 103 and the broker will be forced to buy from the HFT firm at a higher price 104 (big orders eventually always move the market!). The HFT firm has now done a nice scalp, it has bought the shares cheap from slower participants and sold to the broker when prices were higher, making the 2 dollar/share spread he needs to cover his risks and operational costs (co-location, fiber optic cables, fast computers, clever programmers, etc…).
This is not front running! Reading buying/selling pressure based on the public feed comes with a huge risk (you could be wrong and sit on a position while the market moves against you!), HFT firms need to estimate the chance of a big order indicating buying/selling pressure. They don’t know whether the orders are real intents or will get cancelled (hence they will rather react to trades + orders, rather than orders simpliciter). Indeed they do have loss making days! They don’t have any advantage in information (available to everyone) and they don’t play by different rules (no way of jumping the queue). But do they have an advantage due to speed, fiber optic cables, co-location, fast and clever algo’s? Definitivamente! Is this advantage unfair? Depends on what you call fair..
Around the 16 hundreds there were a couple of big companies involved in transporting and trading products between Europe and their colonies. These companies were so big they were owned by hundreds of people and their shares were traded amongst investors. Some people specialised in the mere buying and selling of these shares as news came out about their successes and losses, influencing the price of the shares. News however came out painstakingly slow. Initially one had to wait several weeks for one of the big ships to return to hear about the fail or success stories of the enterprise. Soon, people specialised in trading these shares based on the information that got out, hired people with smaller and faster ships to travel to the colonies and get the news about a company before the ship would have returned. Paying for these ships or the information that came out of them only made sense if you traded a lot and hence your profit depended highly on this information. For a normal investor with a longterm perspective who would only trade their shares a few times a year it would not make sense to pay for this faster information as the costs would be higher than his return on investment. Was it fair that there were people specialised in the mere trading of those shares that were able to pay for these faster ships to get hold of this information before anyone else?
The issue is not technology simpliciter, and it upsets me that Lewis’ his book plays on the fears of people towards complex technology to sell a juicy fairy tale about HFT rigging the market.
Another thing to note, is that while descent regulation is good and necessary, the more complex regulation and expensive compliance to that regulation becomes, the more difficult it is for smaller participants to stay afloat, which causes the big players to gain even more market share and a privileged position. The big players have no issues adjusting their operations to new regulations.
From MC: You are also over simplifying, but more importantly, while you dont want to call it front running of the illegal variety, its still the equivalent of front running and you are making Michael Lewis’s point that this type of HFT is costing consumers money. Those trades could have crossed in a lit market very easily. Without the need for the intermediaries to get a return on their huge capital investment. THose trades could have been completed without the need to “iceberg orders” to hide them. Without the need to have 30 plus exchanges and pools to route orders through.
Dont you think this complications of markets has costs to investors ? Someone has to pay for all that investment. Its not a contribution to investors.
We dont know the math of just how much capital is invested by the HFT players or just how much risk they are willing to take to get a return. Nor do we have a way of knowing just how much risk is introduced to the system by their efforts and their algorithms.
Yes, its true that old school market makers had to take big risks and many lost money. But those risks were a reflection of supply and demand for the shares of stocks they made markets in. Investors paid market makers for those risks, but they knew what they were paying and they knew what they were getting.
And in your over simplification, unless its changed dramatically in recent weeks, exchanges in particular, via order types do put people to the front of the line, or create multiple lines. All so the exchanges can compete as public companies for orders. Do you think exchanges want fair markets or higher stock prices ?
Responding to MC (quoting him from bottom of my post): “You make my points. The algos compete because like market makers of the past, there are guaranteed profits available. Unlike the past the competition between algos create huge risk. Systemic risk. Take out HFT and yes spreads may widen. But so what. Thats far better than having systemic that could impact us far worse than. The flash crash”
But the subtle differences are everything. The notion of “guaranteed profits” is false. Floor traders can, and did, go bust back in the day. Same with market makers. Those guys went against the tide as a matter of course, and sometimes they drowned. Floor traders and market makers live in perpetual fear of informed buyers moving size. The perception of “guaranteed profits” is weighed against the risk they take every day of getting picked off or drowned. HFTs take the same risk. Part of the reason they move so quickly, in addition to competing with each other, is to get the hell out of the way. I don’t think it’s accurate to use a phrase like guaranteed profits when the business model itself contains real inherent risk, along with huge implementation costs and the substantial risk of being competed out of business by one’s fellows. Floor traders and market makers also ate each other back in the day, which is why the weaker ones went under. Same with HFTs. This brutal competition process belies the notion of guaranteed profits and makes the markets function better.
As for systemic risk, flash crashes etcetera, we had all that stuff before HFTs existed. The crash of 87 for example. Or any of the big crashes and panics going back since forever. The old system was not set up to handle crashes — the human beings just stepped aside when everything went under. Systemic crash risk is an age-old market problem relating more to human emotion, monetary policy and economic boom / bust cycles than market maker functions.
Also, it’s a very big step, and a heck of an assumption, to say that getting rid of HFT is worth it just to forestall the possibility of future flash crashes. There are strong arguments, and empirical evidence, that HFTs are doing a better job as liquidity providers than the previous guys did, as evidenced by lower trading costs on net and consistently tighter bid/ask spreads over time. If we really wanted to stop the possibility of market crashes, we could go back to nickel spreads and put breakers on everything. But then trading costs would skyrocket, the i-banks like Morgan and Goldman would go back to making many billions off spreads, whereas the HFT guys are making far, far less via heightened competition, and trading and investing would be more expensive for everyone — with future market crashes still a fact-of-life likelihood anyway. It is necessary to have a better sense of the good HFT provides — which requires taking a hard look at the benefits of liquidity provision — before deciding that the bad is worth banning HFT on light evidence. Again, the car example: If all we heard was the negative side, “get rid of cars” would be an easy sell.
From MC> Every crash has been because of market participants ignoring the obvious and saying that we can’t be afraid of something because of “black swan” risco. That black swan risks are always there. THere are black swan risks to HFT. We cant quantify them at all.
And just because some HFT participants lose money by being willing to take more risk doesn’t mean that the game is not rigged.
A Slot Machine is rigged. Right ? No one hosts a slot machine that pays out 100pct or more. Right ? But if one company wants to pay more rent to host that slot machine, or wants to slice and dice and try to get in front of individual pulls of the slot machine, that doesn’t make the slot machine any less rigged against the “investor/trader/player” Order types, latency arbitrage , etc create a riggged opportunity for HFT players in the AGGREGATE.
All in all, you continue to make my point. The game is so rigged that people are willing to invest incredible amounts of money to play the game. In their rush to play some take on more. Some lose . That doesn’t make the game any less rigged. You have the exchanges doing everything possible to create as many pulls of the One Armed Bandit as they can and to incent more HFT players to come on board. That doesnt make the overall game any less rigged.
As far as the market makers, yes, a lot went out of business. PArticularly as spreads narrowed. But they knew the game they were playing. That is not the case today. No one knows exactly how the game is played or the impact of the risks HFT Players are trading. Both the good guys and bad guys. Which is why there is so much confusion.
And this is before we have seen any malicious players. What happens when people start trying to hack the messaging systems or do the equivalent of DDOS attacks on quotes to create confusion ?
We dont know what happens.
The other issue nobody talks about with HFT is the almost complete lack of system security. When every microsecond counts adding code to check for malicious and malformed messages is a luxury few can afford. The will come a time when somebody figures out how to crash or manipulate the opposing traders systems with a malformed data packet and then things will get really interesting.
A friend pointed me toward this post and asked my opinion. Thought I’d post up my reply to him:
“Front Running” implies illegal activity or violation of a customer relationship. To front run someone means operating on illegally obtained information, screwing over a client with whom you have a fiduciary relationship, or both at the same time. Neither applies to HFT. The information they use is legally obtained and technically in the public domain. Whether the exchanges should provide it is another question. But there is no front running if one is rigorous as to what front running actually means.
The idea of a “guaranteed profit” does not fit empirical evidence – HFTs erode their own profits by narrowing the spreads. HFTs compete viciously with each other to capture a piece of the bid/ask spread, which in turn narrows that spread and lowers trading costs. Technically an old school market maker would have a guaranteed profit if he were the only MM connecting the bid and offer. But the presence of multiple market makers introduces competition, which is why margins are narrowing. Consider: “TABB Group estimates that US equity HFT revenues have declined from approximately $7.2 billion in 2009 to about $1.3 billion in 2014. Looking at recent public data, the profitability of HFT firms in the US equities market has declined, just as the number of players has decreased… If the exchanges, brokers and HFTs are not reaping the rewards, then where is this leakage going? This money is going back to investors in the form of better and cheaper executions, as few if any institutional investors we have interviewed – and we have interviewed thousands – have ever expressed that their equity implementation costs have increased, meaning … trading just becomes cheaper and cheaper. That cost comes from somewhere: market makers, speculators, brokers and exchanges.”
The idea of a “guaranteed profit” also does not fit risks and costs of HFT. The whole reason Knight Capital got acquired is because their algo blew up and cost them hundreds of millions (I forget the amount). HFTs take a lot of risk in the same manner that old floor trader and market makers took risk. Market crash? Hosed. Algo crash? Hosed. HFTs also invest huge amounts in software and infrastructure. What happens if your firm invests $500 million and then your algorithm goes bad? Hosed. There is no guaranteed profit any more than a grocery store has a guaranteed profit because it can mark up the wholesale price. The risks lie elsewhere.
Much of Cuban’s characterization, on balance, can be applied (or mis-applied) to the old systems. If one wanted to critique floor trading, or old school exchange market making, one could similarly hand wave about guaranteed profits and such. But those guys can and did have real risks, and went bust at times, and had general competitive risks from each other that thinned the herds dramatically. Even in floor trading’s heyday, the majority of would-be floor traders busted out.
The liquidity provision of HFT is underestimated. These guys who bitch about the sketchy issues are still overstating the magnitude of the problem — which resides at the margins — versus the beneficial aspects of the core activity. It’s a weighting thing. They are under-weighting the value of large-scale buying and selling activity. For those of us who buy and sell infrequently, it is value-add to have others who buy and sell very frequently, as such that odds are greater that when we want to transact, someone is Johnny-On-The-Spot. If you underweight that core provisional value, the relative size of the problems gets distorted. Imagine a conversation of the relative merits of the automobile — people getting killed, pollution, fuel cost, traffic jams etc — without factoring in the net positives.
The smartest guys in the room (who are non-HFT) are not bothered. Cliff Asness runs.
$100 billion for fees measured in basis points. If anyone should be up in arms at HFT shenanigans it’d be him, as every basis point taken from an institutional money manager’s performance is food out of mouths. But he is chill about it. This is a simple point but one of the largest points of all.
From:MC. Did you read my post? You make my points. The algos compete because like market makers of the past, there are guaranteed profits available. Unlike the past the competition between algos create huge risk. Systemic risk. Take out HFT and yes spreads may widen. But so what. Thats far better than having systemic that could impact us far worse than. The flash crash.
Very good summary, given that it took only 2 hours.
Exchanges don’t regulate because exchanges make money out of volume. A maneira de regular é simples. First HFT should not have information on the order book. Second, all trades can enter a very short “time zone” in which they are randomised in such a way that the flow is not disturbed.
The third way is to create an exchange only for HFT. Let them fight it out as grown ups. Stealing candies from babies is not really ethical.
No one can really know the impact of HFT on price formation. Excessive leverage in any system creates unknown risks.
Before HFT did we not have market makers who took the spread? And is quote stuffing not just price discovery (although illegal) ?
I first would comment that you are either are completely unaware, or completely misinforming your readers in regards to how retail trades would be executed via their online brokerage. In general, with any retail brokerage firm, your stock order will actually never reach an exchange, and never be able to be bought or sold by competing HFT firms. In reality, TD, Schwab, etc. will have sold your order flow to a particular market making firm. These firms get first dibs on your order, and its easy to understand why. Let’s say I want to buy stock A, and being a liquid name, I decide to enter a market order when the bid is 60, offer is 60.02. The firm paying for my order flow will know that as long as my order is executed inside the NBBO, it is a legitimate order. They might try to buy the stock at 60.01 in the actual market place, and then sell me 60.02, and pocket this difference. They might even trade it at the same price and collect the rebate. If they don’t want my order they can dump it on the exchange for HFT firms to fight over. I won’t comment on whether payment for order flow is bad, but it has been going on before HFT.
Furthermore, in regards to algorithms “jumping ahead,” of orders, there is no way to jump ahead of an order in a FIFO market. If I decide to place a bid at the beginning of the day at 60.01, a HFT can’t jump ahead of me at noon because they are faster. Additionally, let’s follow up on your example of jumping ahead of a larger trader establishing a position. If a large trader lifts all the offering orders on a single exchange, there is no way for a firm to cancel those resting orders. Now, after those resting orders are hit, they can adjust the price they are willing to sell that security. But, this is no different than how market makers of the past functioned, or most aspects of business function. If a farmer sells apples to restaurants, he initially establishes a price. If more and more restaurants, or a single restaurant keeps buying his apples, he has the right not to sell apples for the same price. A restaurant doesn’t need to keep buying apples from him, and ultimately, they could buy apples instead of oranges. On the institutional brokerage level, or prime broker level, I would just say that if a broker’s execution is slipping because they can’t keep up with bigger/more savvy firms, it is akin to competition in any other industry.
In regards to strategies, and profitability, I believe you’ve not conceptualized reality in that both personnel and technology related costs could make a profitable strategy actually unprofitable. Every time Google, Twitter, or any website sells ad space, they are making money. It’s always a profitable transaction. However, after factoring costs for servers, programmers, etc., it might be a net loser.
Fragmentation, payment for orderflow, massive indexing/etfs, poor risk controls, etc. are all risks to the financial markets. There are been crashes in ’08 that had legitimate fundamental reasons, and crashes like the flash crash. There are problems that need to be addressed, however, a long-term investor should be more concerned about what amounts to a one bps cost to their investment.
Desculpa. I am not talking about conspiracy theory.
To be sure, I am talking about a big conspiracy theory. Most retails brokers don’t have capital or talents to handle retail orders properly. So they just reroute the orders to an order processor in exchange for order flow rebate.
Almost all the people in HFT I have met have no training in regulation, practically no understanding of fiduciary obligation, or worse yet the regulatory premise of leverage or order submission.
Their compliance officers, of course, have no idea of technology.
So, there have been so many accidental strategies somehow making money everyday. Most of them did not make a big money.
Those who understood why they were making money and exploited the regulatory incompetence made big. As my example shows, the order matching system did not front run client orders, but what happens on the exchange is fronrunning. I believe that this is a breach of fiduciary obligation. Do you think regulators have the equipment to discern this kind of subtle difference? Computer clocks are based on 60MHz crystal. And the time markings are done on different machines.
Again, there are a handful of HTFs that make money as a fully independent systems, but not that many.
The so-called latency arb is certainly profitable, but not a something worthy of $250MM trading capital, unless you know how to create latency when you wants. (I believe creating latency intentionally is a fraud per Exchange Act.)
Knight used to process 1/3 all the U. S. retails orders. Retail brokers route their client orders in exchange of order flow rebate. Knight may match the orders internally or just reroute the order to an exchange.
Hmm. I don’t think I’m getting you. Why would TD Ameritrade be sending an order to a prop-shop’s firm-wide back end? I feel like we are talking past eachother. Or maybe there is somthing I’m not understanding.
No, I am not underestimating the scale of HFTs. My points are.
Completely Independent Prop HFT should abide by Reg T fully and preemptively. And the Reg T applies to even unexecuted open orders. The capital cost is quite high. Layering short orders are practically impossible as all the short orders should have borrowed the stocks in advance. Don’t tell me HFT can borrow stocks in milliseconds.
Most HFTs are working as a broker dealer or just using BD’s regulatory exemption. To utilize the BD’s exemption, the regulatory checks should be done through firm wide regulatory checking system. There is no room for optimization.
Once, a BDs’ exemption is utilized, the system is a BD. As such, the system should 1) maintain orderly and fair market and 2) keep its fiduciary obligation. It does not matter TD America routed the order to the system or not. Once the system sees an order (X) that is not its own, the system should stop any processing its own order till the order X is published by the exchange if the order is routed to an exchange.
Beyond that, I think this is true: “there is so little knowledge and understanding of what is going on with HFT, that I believed that someone needed to start the conversation” because the issue is complex and inherently not reducible to sound-bites or easy metaphors that link up with normal everyday human experience. There is also very little knowledge and understanding of how quantum mechanics works, for similar reasons.
Obrigado pelo link.
It’s not just quotes stuffing! I suspect simular funnyness on the depth feeds also. But this is not an essential result of HFT. It’s a bit sleezy I agree, and infinately simple to fix at the regulatory (even exchange self-regulatory) level.
Also, there are are ways to side step the issues created by quote stuffing.
I’m currently reading the Flash Boys book. And I find it facinating, it fills in a few holes in my understanding, and confirms things I thought… but it’s way over-the-top sensationalism. It’s designed to get you angry, so that you will tell your friends and they will buy the books too.
It’s typical American fear-based media. (I highly recomend it 😉
& gt; I don’t think that most HFTs are that profitable to justify the trading capital.
You greatly underestimate the scale of these operations.
Mark, one item that is missing from all of these arguments is that HFT companies receive a rebate (get paid) to provide “liquidity”.
So while the “normal” investor pays commission to purchase and sell; the HFT companies get paid by the exchanges every time they get hit on the bid-side.
Not only do they make money on the front-running, but also receive healthy payments from the exchange for the illusion of their volume.
I quit trading years ago; when the exchanges overturned “flash crash” trades; meaning I couldn’t even make money when the algos went wrong.
It’s more than fixed – it’s geared heavily on the side of the algo traders (because they don’t have to pay when they make mistakes).
Richard – darthallies… You are absolutely correct, there is no such thing as a truly riskless financial instrument. Even T-Bills have a theoretical risk. However, consider that the (recently delayed) IPO for Virtu, one of the larger HFT firms around, in their pre-IPO disclosures indicated that from January 1, 2009 through December 31, 2013, a total of 1,238 trading days, they lost money on exactly 1 day. Definitely not riskless, but pretty close.
Even if you had the ability to reverse engineer all of the various proprietary algorithms in HFT land, “Quote Stuffing” (explained well in this article: wapo. st/QLKNCJ ) often rendered Thor (RBC’s smart routing mechanism) ineffective.
There are many layers of this onion and I can only hope that many more will now be revealed soon. I’m simply thankful that we now live in a day/time where social media, Twitter, blogs and the like make these kind of secret little evils much easier to expose. From the late 1990’s through the early 2000’s, SOES (Small Order Execution Service) bandits were the scourge of the trading industry. Despite substantial efforts by industry associations to educate congress, regulators and anybody else that would listen, these leaches found a way to expose an order routing system that was created for small retail investors to their advantage, making billions in the process. This practice went on unchecked until the SEC finally woke up in 2003 and ended their honey hole. Not surprisingly, many of the current day HFT firms sprang from the SOES firms.
About optimized regulatory checking.
If an automated system performs regulatory checks by itself, the system should have a separated account with its own trading capital, and probably a separate firm id. Absent these independence, HTF has no legal way to optimize regulatory checks better than firm wide regulatory checks. Obtaining these independence cost a significant trading capital. I don’t think that most HFTs are that profitable to justify the trading capital.
For the order internalization. Client orders may be internally matched against SIP feed abiding by all the regulations except one. For a buy order, if the actual market is down and the SIP feed is stale, the order processor can just match the order (sell), and buy back from the market.
As this is a riskless principal trade, the execution should be marked as such and the client should be informed. I am not sure that those executions are marked properly.
HFTs are self checking systems. HFT orders does not go through firms checking systems.
Who do you think process your orders to ETrade, TD America, Merrill Edge, and so on?
If your broker is selling access to your orders BEFORE they get to the exchange, then blame your broker, not HFT.
& gt; HFT’s order are being sent out without any checks.
My understanding is that is not true. HFT have regulatory requirements too. Although they are free to optimize them.
Let’s say I sent an order of 10K (
$300K) MSFT. The internal matching engine of my broker (or the broker’s order processing firm) will probably just reroute the order to an exchange and send an prop order of 2K MSFT back to back to the same exchange. Which order do you think will arrive at the exchange first?
Just before a client order being routed to an exchange, the order should go through many regulatory checks and the order information should be checked in to a database system (Oracle). HFT’s order are being sent out without any checks.
Retail orders can be front-run in reality though the matching engine itself did not front-run retail orders.
People that says HFT has NO RISK lack imagination. Take the Thor program described in Lewis’s book Flash Boys. They way it’s described it would completely mitigates the HFT advantage RBC had identified. I would content it would be trivial to modify it it not only take away the risk, but game it to your favour. There is no Algorithm, once you know how it works, that can’t be exploited to your gain.
Bottom line for me is that I – a small time investor – got out of the market a few years ago for this reason. I expect The Tax Man to have his hand in my pocket, but not some mystery person/business that controls the topside of my trade. Now I invest only in what I can see and control (right now that’s real estate), which is much different than what my parents and grandparents did to grow their capital. If Wall Street wants to continue seeing dollars flow through the markets, they should protect the system by shutting down this kind of skimming.
Please give citation where exchanges accept money for permitting special order types… I’m assuming your talking about intermarket sweep orders.. price to comply.. hide not slide??
What changed is that the exchanges both delivered information faster to those who paid for the right AND ALSO gave them the ability via order types where the faster traders were guaranteed the right to jump in front of all those who were slower (Traders feel free to challenge me on this) .
There is a subtle difference between volume and liquidity. Just because there is volume doesn’t mean there is liquidity. Very interesting and informative article.
I think Michael Lewis makes a good point in that the HFT firms take NO RISK. Once they invest in the tech, they basically mint money. And if this is the case, how are they ‘participating’ no mercado?
They are simply ‘taxing’ everyone else. Question is, should they be allowed to? And more importantly, can you actually stop them?
In my opinion, HFT provides NO value to the Market or investors. It siphons billions of $$ to a bunch of really smart people that should be spending their time creating something productive. I think the best solution (Mark said this during an interview) would be for companies to require that their stock have a minimum holding period (10 sec, 30 sec, or 1 min). I think this would make corporations look like they care a little more about their long term investors – the people that vote for their boards. I still think Insider Trading is a much bigger problem and find it unfortunate that the SEC (with it’s limited resources) has to spend time dealing with HFT and not the bigger issues.
Good recap of the situation Mark, however one important point that should be highlighted is the complicity of our for profit exchanges. While HFT’s reap the rewards of dashing in and out of the markets in the microsecond world, the exchanges are the ones that have made it possible by providing premium co-location services and special order types. While these “services” are available to anybody, most of the investing public does not have an extra few million/year to drop a server into the exchange’s data center… ensuring their fiber optic cable to the exchange’s box is a couple feet shorter than the competition, thereby giving them a couple picoseconds advantage. The exchanges, in lockstep with HFT, have reaped these rewards through these co-location fees and increased volumes. I feel that this point, should it ever come to be widely understood, could truly shake the confidence of the investor. Not only were the thief’s stealing, so were the police… just in a different “kinda sorta” caminho.
Having been a sell side trader for the better part of two decades, I’ve had to watch in horror as my customer’s orders have been screwed and tattooed in the name of the HFT’s providing “liquidity”. While my customers are typically large money managers, mutual funds etc, in the end the real customers, those that are paying the price, are the investing public. If a mutual fund manager pays more because an HFT sniffed out his order and scalps a few pennies here and there, so does George and Linda in Dubuque when they invest their 401k in said manager’s fund.
One final point and I’ll go back to my hole… One of the primary reasons the big banks have been slow to display any kind of outrage is because they’re part of the problem. Every major bank in the US (GSAM, Citi, BofA etc) has it’s own black-box HFT style group. Tough to point the finger at yourself!
Here’s my analogy for people who don’t really know how trading and this issue really work:
You go to the grocery store to buy milk. As you walk in the door, the grocery store stamps a sign on you saying “Here to buy milk.” That grocery store has someone paying them to put that sign on you. Then, as you go to get that last carton of milk, someone grabs it before you do. That person then turns around and sells you the milk for more money than the store was charging.
First, there is nothing wrong with speculation in the market…everyone can and should have their own motivations for being involved and that’s what “makes a market”. However, the playing field should be level and selling access to others’ orders is effectively inside information.
I submit one very simple suggestion for leveling things in what I feel is a very fair way:
All orders at a given price are treated on a time priority. That means if I entered my order to buy Apple shares at the same price as you but I did so earlier, then I get the first execution at that price, no exceptions. The “jumping the line” issue that we have now is the real unfairness and is what is allowing the system to be gamed. If someone wants to pay up to the next price level that is a totally fair part of the market but at the same price time priority across all exchanges should exist.
As this debate progresses, also don’t lose sight of how the exchanges are making their money (Putting that sign on the milk buyer’s head). Special access, esoteric orders, etc are all major revenue streams and the HFT crowd is the cash cow to them, so changes will be resisted.
Full disclosure, I spent 8 years as a NASDAQ market maker, another 6 as a proprietary trader, and the last 9 in equity research and portfolio management.
Brilliant, concise explanation. Just leaves out the cause, which will also answer many of the questions posted. HFTs came about and thrived as a result of stock exchanges converting from private memberships to “for profit” publicly traded corporations. That is when and why they stopped servicing the customer and began exploiting them in an effort to create and exploit new revenue streams. Had exchanges maintained a fiduciary responsibility to the customers they were built to serve instead of the shareholders and investors desperate for constant growth, this debacle never would have happened. The model of a “for profit” exchange is an unsustainable one as it is impossible to continue to exist when you put investor profits ahead of customer fills.
Anyone remember rules 127 and 72B? A specialist could sit on your super dot order for more than a minute before giving you a report. In some issues, I couldn’t get 500 shares without paying up a quarter on every trade. Or how about the Nasdaq lawsuit in the late nineties. OTC market Makers were colluding with each other and fading whenever even one of them was called.
Im not defending HFT, they put me out of business as a floor guy, but I don;t see where there is any breach of fiduciary going on here. The fact remains customers have it better than they ever have had it before, and when you eliminate incentives to provide liquidity, it will worsen.
As for the transaction tax solution, consider making it rebatable if the position in question is held for a year and a day (the same standard that determines long-term versus short-term capital gain). We already grant long-term positions an extremely favorable tax treatment, so we have already made the political judgment that this is beneficial to society. Does the HFT issue fold-up shop at that point, or does the game just move down the street to someplace else? (honestly have no idea)
Great post, the reality is that hft and the algorithms have changed the structure of parts of the stock market. No value what so ever, in promoting the purpose of the stock market, which is to raise equity capital, and realize liquidity for your equity. If you choose to participate, know what you own, and calculate the value of that piece of equity. This is why alternatives to traditional exchanges are popping up, ie second market.
Good post Mark. I’m not trying to challenge you, but rather share my experience.
What HFT trading seeks to do is read the order flow, something that has been happening since the advent of the bucket shop. You point out that it is algo’s now trying to do this, and not necessarily blatant front running, so there is inherently risk involved. Risk of any kind deserves reward.
Reading the order flow used to be something we paid for in the form of seat leases, stand on the floor, and know who is trading what? And when they are opening or closing? etc..Now its a matter of paying the exchanges for the information as fast as possible and having an algo interpret it for you. Exchanges are thriving as a result.
Nothing has changed, just how, and who now has the edge.
If you really want to catch a thief, look into the structured products being marketed through major wirehouses, price them out relative to listed options, then tell me how these firms keep any of their clients or stay out of hot water with FINRA.
Given my earlier post, and on further reflection, due to the need to promote “fairness” in all markets I propose that the NBA force Mr. Cuban to buy Kevin Durant to make the Western conference more exciting and equitable in the interest fan welfare.
First some history since no one has explained how knowing an order beforehand makes someone “billions”. It doesn’t! Front running an order doesn’t make money unless the order is so large that it moves the market in the front runners favor. Front running a client order is illegal, and technically this is not what is happening, since the HFT trader (who might sit on another floor of the brokerage) is neither aware of nor cares about a client order.
HFT grew out of the need for arbitrage between a stock index and the underlying securities. If there was a misalignment then the market (i. e. everyone) suffers due to misinformation. The problem was really big when stock prices were priced in a 1/8th’s (i. e. 12 1/2 cents) spread. Moving to penny pricing allowed dealing firms to undercut each other which benefited all investors. HFT firms have arbitraged this spread away and are shutting down as a result. Here is why….
In order to make this arb work, on say the SP 500, a dealer needs to buy the index (say the futures contract) or typically have an existing long market exposure through owning all the stocks contained in dealer inventory and then sell every 500 stocks at exactly the same 1. time; 2. appropriate price differential; and 3. appropriate volume to rebalance the portfolio perfectly. AND later buy back every single stock in the same fashion. This is why there was a race to faster computing power and execution speed and why it benefited the market that regulators allowed it. Tighter spreads means better information and pricing value for everyone.
BUT, here is why some in the industry don’t like it…execution speed has gotten so fast that it exceeds that of the exchanges. A dealer can send an order to determine bid/offer and volume depth and then cancel it before it executes; this is called “pinging”. This is done to ensure that all the orders placed can be executed at the same moment and volume to ensure a perfect risk-free arb. Pinging ensures that the market is properly aligned, since there is no order executed unless there is profit to be made, but is this fair? Naturally markets have fixed bandwidth which results in sharp price movements when everyone wants to execute at the same time, such as during stock market crashes or dropped calls on cell phones. Naturally HFT arbitrage benefits hugely during these periods, and as should be clear by now doesn’t cause it. Since the process ensures tighter spreads and accurate pricing for all markets regulators are having trouble how to respond to the “unfairness” of certain participants who know more and sooner than everyone else, but that the nature of the free market in information. To argue otherwise should require those, like Mr. Cuban, to give up all their money to the poor and needy because they didn’t work hard enough or smart enough for it.
And just to get in another dig (since you have read this far) the practice and the complexity of the problem is hard to explain to an ignorant media and public that prefers sound bites over some hard thinking….IMHO.
To Valerie Alexander and others proposing a tax. So, your solution to the problem of firms shiphoning capital from a pool of capital that is supposed to be available for investment is to have it instead shiphoned off by the government…. brilliant!
HFT firms sends out massive amount of orders to the market.
1. Do the orders abide by reg T?
2. Are the shorts really borrowed?
1000 share each over 1000 stocks costs 15MM initial margin. To send out orders over 10 ticks, the firms should have at least $150MM trading capital sitting on their account. Let’s say these are buy orders. Now, the firm cancels bottom tenth orders and send new orders on top of their best price. If the firm sends out the new orders before cancel confirmation, the firm needs extra $15MM. When the market is fast, the firm may need to send out 4 layers of orders above the previous price, which means extra $60MM. Now, what happened to their sell orders sitting above the market? If some of them were executed, let’s say two ticks, that’s another $30MM.
Just to handle reasonably fast market, HFT firms need $250MM trading capital. The real question is whether the capital is committed or just booked via other assets which the firms do now have the ownership.
I am sure that the HFT firms are taking advantage of BD exemptions, but the exemptions are only for making orderly markets not for prop trading.
The Risk of HFT? What if all the orders within 10 ticks are executed? The firm should come up with $150MM out of somewhere. For the firm, it is better to sell them and take loss of $5MM rather than ending up in settlement failure, which guarantees FINRA investigation. Look at Knight, they could not settle the trades, doesn’t they?
If poker were legal, would the FTC allow a poker site that lets one person see another person’s hand before they bet? Of course not, because it’s not fair to everyone involved.
Would anyone play on that poker site? Of course not, because it’s playing with a bunch of cheaters.
When that game is your whole economy and the financial well being and confidence of your citizens, it’s up to the government to weed out the cheaters especially if it’s out in the open and affecting consumer confidence.
Do you know people that would like to play in a poker game where you can see the other person’s cards? Absolutamente! It’s not honest but there are people ready and eager to steal from the unsuspecting. So that would drive normal investors to seek out these types of advantages because they realize they have been on the short end of the stick and want to change teams. When this happens it will exacerbate the situation and lead to a future of all cheaters who are no longer playing poker and the market will die.
If cheaters go unchecked they get greedier and greedier because they don’t really think they are cheating and nobody is calling them on it.
This boils down to the importance of consumer confidence for our US economy.
Balazs, Cross exchange arbitrage has always existed and I have no problem with that. What you’re describing is front running enabled by the exchanges by selling the ability to execute the trades ahead of an order hitting the book. The fact that only part of the order is front run because some of the order already hit the book on another exchange is irrelevant. That isn’t cross-exchange arbitrage where two trades are made with two separate counter-parties – each at least standing *a chance* of receiving the price that they expected. In your example, it is almost assured that the one counter-party being front run won’t. Payment for order flow is another opportunity to front run whereby an entity gets to choose how orders are routed.
Of course, all of that assumes your assertion that cross-exchange front running is all that’s occurring and that orders aren’t being made available to a select few before they hit any exchange. I wish I shared your faith. Even though we are told that ‘Flash Trades’ (where order information WAS being bought ahead of execution on any exchange) are no longer permitted on major exchanges, I’m not so sure.
Reblogged this on The Green Pulpit and commented:
Barry Ritholtz wrote: “Why anyone is allowed to see other people’s orders, than front run ahead of them defies explanation”.
Thank God! I was beginning to think I was the only person asking this question.
I find it ridiculous to see so-called, ‘simple’ solutions being suggested such as applying a transaction tax and introducing a 0.5-1 second lock on orders or calls for new legislation etc.
Front running is already illegal.
Trading on advance/insider information is already illegal too.
What is happening here is akin to allowing a chosen few to see other people’s cards in a Poker game – and then having to listen to venal and corrupt individuals trying to defend it as being a good thing for everyone.
There is NO good reason in allowing anyone to see anyone else’s orders before they hit the book. The fact that Specialists have been able to do so historically is not an excuse to do so in the modern, technological age. They did it that way because there was no better alternative at the time – not because a chosen few being able to all orders before they’re matched was a ‘good’ thing for everyone. HFT front runners pay the exchanges to be able to do this. So, what happens if they don’t pay? They don’t get the information. Which means the exchanges can stop this nonsense in a heartbeat if they wanted to. Actually, we don’t even need them to ‘want to’. We already have laws against front running and inside information. Those laws do not excuse those practices just because they occur at sub-millisecond speeds or for transactions of just a penny.
I wish people would stop considering other alternatives as a remedy. It only distracts attention away from the real problem – and therefore the only solution to stop the cheating and rigging in this way.
If a chosen few were allowed to see your cards in a Poker Game, would you really be calling for a ‘tax’ on them instead of just demanding that the ‘pay-for-view’ stopped?
marmun1, it’s not that banks see your order before you place it, but they see it in the first exchange, and then they can frontrun you to other exchanges *assuming* that the first exchange was only part of the order.
Great writing, but I’d suggest its like anything unknown, it seems scarier than it really is, and even if you assume HFT is siphoning off billions (for the sake of argument), that is nothing compared to the trillions of wealth generated in the markets. The drastically declining returns of HFT funds seems to disagree with that hypothesis anyway – IMHO they’ve created so much pricing efficiency (or scared retail investors away from the markets), they’ve caused their own demise and are now largely competing against each other.
This isn’t to say anything is right or moral – just that its not as great a problem as people[/media] are making out. The “evil bankers” are just geeks who love math, code and an infinitely tough challenge – and majority are not making millions. The average quant salary is far less than the west coast. They’re doing it for love of the math & a challenge.
Like all things (cars, planes, jobs), the markets are becoming automated and, better or worse, *high-frequency-trading* is a part of that that will stay. There may be a few hitches along the way (front-running), but the value destroyed and corruption by humans is equally as bad.
The upside from having emotionless markets could bring stability for everyone. Better to level the playing field and let everyone invest with algorithmic precision. Maybe by 2100 the markets will be perfectly correlated to the risk-value at present moment in the business the ticker represents >> Someone buys an iPhone, and AAPL instantly goes up 0.000001c.
Mark, you’re right. Although there is a considerable amount of information about HFT, there is a lack of understanding of what is going on with HFT. And yes, the conversation needs to continue (you needn’t worry that you have to start it; a number of experienced traders, market structure analysts and academics have been doing that for nearly a decade.
This may help the conversation, written after the CBS 60 Minutes segment (not by me). It’s a wee bit longer than your 2,571 words but based on nearly 30 years experience watching/researching/analyzing the evolution to automated trading: “No, Michael Lewis, the US equities market is not rigged.
It’s been quoted for the past four days by major media (NYT, WSJ, CNBC, FT, Bloomberg, LA Times, Barron’s, AFP and MarketWatch, on and offline and TV).
Hope this helps….
I used to build the HFT networks for a large financial institution (Fortune 50 sized). Your information is for the most part right on. But as others have noted, the vast majority of investors should be long term investors, where the HFT game really shouldn’t have any impact. Yes, the flash crash phenomenon is a major issue, and needs to continue to be addressed by the SEC. Some of the recent changes they have instituted to prevent flash crashes help, but don’t go far enough. Now, has HFT killed the day trader? Absolutamente. Day traders are playing a rigged game against a stacked deck.
The larger issue, not really addressed here, are the so called “dark pools” where a lot of very suspect transactions occur daily, and to date are perfectly legal. The fact that the SEC allows these transactions to occur with little to no oversight is mind boggling.
Mark, I bet no HFT trader will seriously challenge your assessment. Everyone knows that this kid of front running is exploiting a loophole in the system and adds no value. Who wants to voluntarily put lipstick on a pig… Hopefully it’ll be one of those “it was nice while it lasted” type of thing for them.
diyinvestorsource, your logic is like saying that cars are so much safer than 50 years ago, so we shouldn’t care about the sudden acceleration in Toyotas. It’s true that Joe Shmo selling 50 shares of MSFT is going to be affected only minimally. But what if the mutual funds in his 401k or his company’s retirement fund gets nicked 1% per year. The book mentions an alleged $300MM per year loss for a $9B fund, which is 3.3%. That is a lot if you start to compound year after year, even if only half of it is true.
While front-running is making it’s most of money off of big trades, and so not small investors, related HFT strategies like quote stuffing and stop hunting can crush the individual trader. Artificially tripping a stop-loss on an out-of-the-money option can be worth a significant chunk of the total position.
The negative impact you describe is dead-on and hits anyone that decides to go the mutual fund route, which is most small individual investors that shy away from individual small to mid-cap stocks with their lower volumes and lower liquidity and the attention those investments demand. They already pay a price to go the funds route – the funds loads (direct or indirect). HFT is a greed surcharge assessed by the HFT traders that has to impact the small investor in the pocket book and undermine confidence/trust in the markets. Time to bury gold coins in the backyard. 🙂
Breaking down some of the ways this costs you money:
Quote Stuffing – which should be illegal and IS illegal when a person does it (it’s running a ‘boiler room’). Aside from perpetuating a fraud and slowing things down the way Mark described, it slows things down in real time as investors waste time chasing phantom trades; trying to execute on orders that don’t exist, which prevents them from actually doing the work their clients pay them to do.
Front Running – many of the posters in the comments here seem to think that it’s simple price shaving, it’s not. They are straight raising the price of every large trade. Most trades don’t occur at a single price point, they occur at a range. If you want to buy 10k shares of Apple and are willing to pay $600.98 – $601.00 per share an HFT will see your buy order, buy up every share available at your low end and then sell you it’s newly owned shares at your top end. They effectively cost you $200. Como? Because they didn’t even have 10k shares of Apple to sell you. They only bought the shares a microsecond before you so they could sell them to you. If not for the HFT you would have bought the 10k shares from the original seller for $600.98, but because the HFT bought all those shares out from under you you have to pay $601 to get them from the HFT.
They added NO value and no liquidity, all they did was scalp you.
And this too is illegal when a person does it. It’s trading on inside information.
Small Investors – but what if you’re just a small investor, or a simple day trader? That’s fine, As long as you keep it small and do all your own investing. But most people don’t run their own 401k, IRA, or other investment accounts. All those pooled accounts are big deals and they all have to deal with HFTs which means the ROI of any of your pooled investments are losing a % because of HFTs.
In answer to why the exchanges don’t regulate it, because the HFT firms pay the exchanges for access.
Good stuff Mark – these guys clearly have exploited a hole that needs plugging and they’ve been making a killing while nobody has taken action. They’ve also set themselves up as the great scapegoat when the market does crash…everyone will point their fingers at them and the Fed will get spared while they’ve been the biggest market manipulators of all.
Ok so we agree that this is a really small impact. And only on transactions. Lots of trading = high HFT impact (relatively speaking, it’s still pennies). Little trading = low HFT impact. That’s one point for long-term holdings. As for the crash in 2008 – anyone who was in stocks in 2007 and needed to spend the money in 2009 needs an idiot’s guide to investing. Even at the best of times that’s asking for trouble.
Someone who was in stocks in 2007 and didn’t look at the market until 2012 never knew anything happened. I let others play the short-term game. There is only going to be one trader that is the fastest in the world and they will hurt everyone else who is playing that game. There are millions who have made and will make a profit from long-term holdings. The flash crash? I would have loved to buy into that since it was cheaper than the day before or after. But I’m way too slow because I’m just not paying attention most of the time. I only take advantage of old-school slow crashes.
So if HFT imposes a tax of 0.0125% when I buy today, and the same when I sell in 50 years (ok maybe it’s been regulated out of existence by then). And if all I know between now and the day in 50 years that I sell is that the companies I own keep making profits that get to me eventually, why do I care? Trying to beat the HFTs at their game is a great way to lose. Doesn’t matter to me. I’m patient. Playing a different game is how you win.
People including you are talking about a 1c/share tax to slow this down, which is a far bigger impact than the HFT problem. Whether that cost goes to a trading company or the government it’s the same to me – too small to care.
You can call that rigged, but investors have 99 problems and HFT ain’t one.
The problem isn’t so much HFT but rather the proliferation of trading venues and their intended purpose. For years, stocks were traded on the NYSE and then Nasdaq with no issue from a market structure perspective. Reg NMS changed this. Counter-intuitively, having more options available to you in terms of places you could buy or sell has now become a bad thing (but what about CAPITALISM you might ask). For profit public exchanges have created two tiers of investors by allowing direct feeds that give an edge through the various means described above (quote stuffing, flash pricing, etc). The second tier is the American retail investor and even large banks in many cases that is continuously being skimmed from getting a better price and shown fake liquidity. That’s not capitalism, it’s stealing. For a small investor, do penny’s on a trade matter? For a large bank, do millions of dollars matter? The answer is, of course. This is America. We don’t trade our markets like we spin a wheel in Vegas. For our stock market to drop 10% intraday due to the above mentioned issues (stub quotes), is unacceptable. This wasn’t a “glitch”, it was predatory trading and it’s something that affects every investor. The real dilemma is how little is known to fix the problem. A resposta & # 8211; a not for profit exchange. It solves every issue mentioned and would foster a fair, capilist market, that we trust. This is something that I plan on devoting quite a bit of time to. Get in touch on LinkedIn:
So if buying, why not just place a limit order which requires an exact price? Why would it matter if you got a partial fill as stocks normally bounce around a bit at any price. You may not get filled immediately, but the vast majority of the orders you placed would be filled at the price you specify.
For me, a novice investor, the best part about posts like this is it helps people to understand there is a huge difference between computer-aided trading and what it’s done for spreads, along with switching to decimals (what the hell is a steenth?) and the skimming or scalping that the front-running does to generate millions of dollars of profits for the HFT houses. I’m a capitalist and applaud when someone invents a products and makes a huge score – good for him or her. But this HFT skimming practice helps no one. It adds nothing to the market, facilitates nothing, adds no depth or liquidity, provides no service and solves no problems. The guys who run these exchanges remind me of the stories of when the mob was getting a one dollar ‘tax’ for every window installed in the city. It only served to make the crooks a little richer, one window at a time.
No wonder unemployment is so high, the capital markets have been taken hostage by Banksters who’s desire is to tax every dollar of capital investment. A healthy system would have Financials 5% of total GDP, where-as today it’s closer to 45%.
Break-out your pitch-forks, storm the barricades of the Bankster Castles, and string these rapacious rascals by their necks from every street pole across the land.
Charge Pay-per-view for front-row seats the reckoning, and we clear the National Debt.
Reblogged this on nwuptick.
You are mostly correct. 6b is wrong however. HFT are very involved in small illiquid stocks. The game is different and the spreads get wider. The gaming is far more pervasive than the big stocks, Orders for 100 shares frequently get front-run. One tactic is to partially fill to an odd lot order and becoming invisible to the market. The biggest reason for this behavior is broker internalization of order flow, and showing HFTers orders beforehand to get “improvement.” Instead of getting a fill for .0001 better, you don’t get any fill at all as the HFT uses your flashed order as a signal to do the exact same trade you wanted to do except sooner. Taking the displayed liquidity. What is most ironic is that brokers like to talk about price improvement statistics, and only count executed traded, while trades that never occured because they were frontrun are conveniently left out of their statistics.
Valerie, that creates a disincentive for investors to invest in companies with a low share price. I can buy 1 share of AAPL and pay tax of .001, or a bunch of shares of a cheap stock and pay significantly more.
I also see this as a larger problem of the markets becoming a gambling device or in the case of the Fed, to create a ‘wealth effect’ that distorts the stock market as an economic indicator. Wasn’t the purpose of the stock market to allow entrepreneurs to raise capitol to invest in their companies? Aren’t the HFTs essentially acting like rats in the granery?
Nice post and great analogies. Broadly speaking, while I think front-running (the heart of the HFT discussion) is morally wrong it is capitalism to the extent it is not deemed illegal by the SEC as insider trading. For long term investors – preferably in small cap stocks – the financial “impact” is negligible. However, the potential for further distrust in the stock market (the life blood of the US economic engine) and Wall Street is irreparable.
A bit of history…dating back to the infancy of the stock market, NYSE market specialists in theory traded on information related to incoming buy/sell orders – i. e., they sold the “ask” price and bought the “bid” price which used to be 1/8 pt apart. In many instances, these market makers knew they had a willing buyer when they bought stock from a seller – thus “front-running” in the loosest sense. Further, all trades had to go through them as they were the market specialist in that name. Over time, this bid/ask spread declined as markets began more fluid and transparent. However, the BIG difference is that the old NYSE market specialists of year’s past took real risk – they held stock overnight, had real capital at risk and where required to always provide a market (i. e., a bid) for a stock.
The issue I have with the HFT firms of today is that are front-running purely based on speed and seeing other buy/sell orders before they are recorded as “market trades” on the ticket tape we laypeople see on CNBC and Bloomberg. They are gaining an advantage without putting any capital at risk or providing incremental value or liquidity to the marketplace. They are merely skimming off the top. And under my moral compass, that is simply wrong.
Unfortunately, this will be nearly impossible to police unless you slow everyone down and distribute trade data at the same speed (albeit this is geographically challenging given co-location). One could suggest bringing trading back to one central exchange maned by real people…but there would still be dark pools. One could suggest taxing all trades a penny…but HFT would still exploit the advantage of speed (but I do like the idea of additional tax revenue being raised to help regulate the trading industry). One could suggest bringing back wider bid/ask ticks…but HFTs would again exploit the market based on their speed until the “real” market data caught up.
At the end of the day I’m not sure what the right answer is…public/long term investor behalf will ultimately drive the outcome. The one positive is Michael Lewis’ new book at a minimum has brought it the forefront of our minds.
Mark & ​​# 8211; love Shark Tank…American capitalism at its best!
Jeff and The Tuna, I take it you are saying that the exchanges permit this despite its harmful nature because they are effectively taking a cut of the HFT profits. This is certainly possible, albeit less likely than not. But given the competitive nature of exchanges, if this is true those exchanges that permit the activity will be at a competitive disadvantage and end up losing more than they gain in the profit split with the HFT because they lose listings and volume.
Now of course it is possible that this won’t happen because of some collective myopia, but the more likely result is that if this is unproductive it will sort itself out without the SEC promulgating additional regulations.
Once you go down the road of regulation you don’t usually get perfectly optimal regulation. This is why people should have to tell a more plausible story of structural market failure before invoking the cumbersome hand of the regulator.
In short, this piece makes a better case for why exchanges are likely to restrict this activity than it does for its (admittedly narrow) proposed remedy of escrowing algorithms with the SEC.
Perhaps the scariest thing here is that an investor of Mr. Cuban’s caliber is only kinda/sorta pretty sure he knows what he’s talking about in regards to HFT. Not meant as flattery, but guys do not amass the resources to buy NBA teams without having pretty good investment instincts and thorough investment knowledge.
One point Mark alluded to but could have explored further: people are *risking* billions of dollars for a chance that place in the front of the line, and given the open warfare between HFTs, there’s no guarantee the person rigging the market today will be rigging the market tomorrow. Some of those billions sunk into fiber optic cables and algos will go to waste.
But the most important point here is that the securities markets exist to enable people to invest in companies they believe in, or to speculate on the possible future values of securities, providing vital liquidity in the process.
HFTs appear to do nothing to enable this investing process, and therefore there is no reason for the exchanges to allow it, except for the short-term profits they reap while taking on unknowable risks.
Mark it is great of you to clarify in some amount of detail (much more than what is allowed to be publicly discussed on financial news networks) regarding fraudulent aspects of the trade. The mathematics behind HFT varies from very simple to complex artificial intelligence driven algorithms especially on the millisecond and tick level and the MAJORITY of these algorithms are truly providing liquidity in smart ways that benefit the ordinary investor. I don’t contend this. You did not mention the prolonged effect cross exchange arbitrage and darkpools have had, but I thought to touch on it a little bit here; you see the greater the fragmentation of the market because of different order types, the greater the proliferation of dark pools and other HFT pools that can game it from different angles. You’re right when you say the average person isn’t concerned with his investment and the investors aren’t going to sound the alarm because there is too much at stake with retail outflow in recent years. But the retail guys have less liquidity to contend with today in the lit market and are soon coming to the realization of what’s happening under the hood. Day traders are unnecessarily and very swiftly stopped out of intraday positions only for the price to revert seconds or minutes later, almost levitating back to their original prices. I have seen the most out of the money stops get hit on an intraday basis like it was an orchestrated panic. It is unnerving for other people to have that kind of power in the market, or for there not to be enough ‘real’ liquidity chasing ‘real’ price discovery. The average guy is catching up, but in many ways, he’s got no alternative way to express his opinion when all the exchanges seem to dictate the rules of the game, watchdog-free.
Welcome to the fray. I started blogging in 2010 at pointsandfigures because I was upset with what was happening in markets. It wasn’t electric vs open outcry, or speed, or anything nefarious. It was market structure. Totally mismatched to the way we traded. It creates an unlevel playing field. That’s not capitalism, and it’s not how America operates. If we continue, more and more Americans will lose confidence in the free market. When that happens, we are at risk to lose our way as a nation.
This read more like the low-latency part of HFT. That is mostly front-running of large institutional trades. It does impact the little guy indirectly in as much of that which is being front run is likely their pension/401k/etc. Just because they’re not trading it, doesn’t mean they’re not affected. There’s also the stop hunting algos which don’t require a low-latency to function. The algo will fire of thousands of orders and immediately cancel them in hopes to trigger the hedges or trailing stops of other traders. When those stops get triggered, a cascading effect occurs and the HFT algos monkey hammer the market. You can tell the stop hunting algos as they produce a square wave pattern on the millisecond charts. This is a special privilege as well.
What if the entire market and its clunky decimal place price levels were to fall victim to the slicers and dicers? If onion skin thin profits on massive volumes are justification for stealing pennies then why have bankers who have harvested “lost decimal points” been prosecuted while these “titans of wall street” are free to boast about their “innovation” (read: we have 50 ways to rip you off and enjoy discussing every one over expensive cocktails at restaurants where you – the little people – cannot get a reservation)?
Obrigado, Mark. It is interesting to note that this game has been played throughout time, just at much slower speeds. As a former equity market maker for a large bank in the early 2000’s, I was endlessly frustrated dealing with similar fruntrunning. Back then, if you wanted to trade with other dealers with a size larger than the SOES amount, you had to basically send a message to the other dealer with the size and they had to respond. If you sent 50,000 to buy to Knight Trading or any other wholesaler, they would delay their response while they bought everything in sight above them. Then they would sell it all to you all 1/16 higher. Thats how they made money. Seems like ancient times looking back on it now, but that much has changed over the years. SuperSOES went a long way to correct those issues. Now the same game is being played, just in milliseconds vs 10’s of seconds. The real difference as you note is the unknown effects of the super speed algos on the markets in times of stress. There really should be a way to prevent the algos from seeing any order size larger than their offer or bid.
Mark when you keep saying that the market is rigged you’ve going to give people the idea that they put in $1 and get 50 cents back every time. Or maybe just that it’s a slow drain like playing blackjack for too long. And I’m sure you know that. But what you describe is more like adding a commission of 1 penny on each share you buy. Sure it would be nice if that was cleaned up.
On the other hand we’ve gone from paying commissions of $300 for every trade to getting them from $10, $5, $3, or just plain free. Even for big traders that don’t even notice their commissions, this effect would have to be so small that it’s just a tiny blip in the risks they’re already taking. I just don’t see anything that points to this being an issue of market structure that even comes near the scale of the normal risks that every investor has to take in the market let alone a lot of things that have been going on for the last 100 years that were just as questionable. Those didn’t destroy the market. Once in a while they created great buying opportunities for investors who were prepared.
There is a very simple way to outsmart the HFTs. Be a long-term investor. The less you trade the less they can take. And the longer you hold the faster they lose interest. Unless you’re trying to play their game and complaining that someone got a small edge over you, you don’t need to follow their rules.
Did you read all of it. i said a specific part of it was rigged . And i didnt say it was anything but pennys or less. And being long term doesnt out smart anyone. All the risk impact the performance of your portfolio. Being long term didnt outsmart the tech crash or 2008 if you had any needs for those funds prior to the years and years it took the market to recover.
and electronic trading and decimalization gave you the savings, not HFT.
From one idiot – Muito obrigado. I don’t feel quite so idiotic now.
Please advise on this video, it involves Forex trading and algorithms, not HFT by definition but short term positions:
I trade the indexes, futures and commodities every day. How things have changed…years ago things liked value propositions, competency, management, etc derived a value; today, it’s who gets to see my orders, very quickly buy and then sell me the stock. It’s turned into a very large casino with some social interests “guiding” the market and NOTHING to do with the company’s. So with HFT, what is the value of all the fundamental analysis, Cramer and CNBC?
Great post, Thanks for the incite. I appreciate that you take the time to research and explain all the angles.
It’s been going on since the buttonwood tree and trust me it’s cheaper than floor brokers they used to front run marathons now sprints and the mkt needs the hft bids when it crashes! the hustle never ends i was born into it so i know first hand this is nothing!
Mark, To me it is all about a somewhat level playing field. If my Mutual Funds,401K Funds, and TD Ameritrade are all using the same High Speed Trading system then we all have a fighting chance at benefiting from HFT. The reality is we don’t. Only the well financed firms that can afford the fee’s and systems that allow then to collocate and get the sub millisecond advantage will make the money and manipulate the market. We no longer have pension structures that are independent of the stock market. All of our retirements accounts plunge most of our hard earned savings into the stock market in hopes that it will be on an up swing when we actually retire. Not always. Asked those that tried to retire in 2007/2008 etc..
The converse of HFT that is even more bizarre is that if a Market Maker shorts a million+ shares of stock, the DTCC (Depository Trust & Clearing Corporation) gives them THREE DAYS to locate the actual shares of stock. If they don’t find those shares or it runs into many days they can just cancel the trade and no harm no foul!.
So you allow both HTF at sub milliseconds and then Days and Days to cover if you are pumping phantom stock.
Neither should be allowed. Pick a reasonable time, 1 second. They have to actually locate the shares within that second whether it is a buy or sell side order and then charge 1cent per transaction and fund the SEC so it can keep everyone honest.
Douglas, the short answer is that Exchanges are making more money allowing the HFT’s to use their pipes. In turn, the Brokers / Market Makers are being paid on trades to use certain exchanges instead of paying a flat rate or fee. This has upended the dynamic the exchanges have historically created, the overall effect of which makes the market that much more ephemeral and opaque. In my humble opinion, this short term view taken by Wall Street does not lend itself to the long term overall health of the markets, foreign or domestic.
It seems there’s a very easy solution to this, and it would be good for the economy on all levels: add a trading tax of .001 per share sold. This means if you move a million shares, you pay $1,000 for this tax. For a deep-pocket market investor selling a million shares of something, that $1,000 is barely pocket change. A rounding error. For the average market investor, selling maybe 20,000 shares at a time, it’s an added twenty bucks — far less than the commission on the sale. For the HFTs, it would grind their business to a halt.
Since the purpose of the markets is to provide companies with access to capital and provide liquidity to investors (thus encouraging them to provide the capital that companies need), not to create trading schemes to make number crunchers rich through algorithmic trickery, this would solve all of those problems. And, the tax revenues could be used to provide further security for the markets, or to replenish pension funds that were wiped out by dicey derivatives.
Why anyone is allowed to see other people’s orders, than front run ahead of them defies explanation. How the SEC allowed the Exchanges to stop serving the public interest is mind boggling.
Douglas - the exchanges like volume. Volume is a huge part of their business. They, of course, need to provide stability, but the money is in the volume.
Mark, this is a very nice summary. The only thing that I want to comment on your post, is that in reality algorithm design and development for HFT its not really that complicated; and you certainly don’t need the smartest people to implement a successful one for a given market niche.
This is the most informative description of HFT I’ve read. Kudos.
Wasn’t HFT the cause of the “Flash Crash” in 2010 also?
I’m an equities and futures day trader, and can say I absolutely see the repercussions of the HFT bots in the markets. The strategies that used to work pre-2008 are now COMPLETELY OBSOLETE. The main way I make money trading stocks is by staying away from anything that averages over 10M shares a day. In the futures market, we have to make sure we only trade when there’s panic or extreme greed. Otherwise we’re just fighting algo’s, which is a losing battle every time.
Mark, thanks for bringing this topic to light!
Here’s a video of me talking about what I think it takes to succeed in the markets today – youtube/watch? v=4rUhj4T1XR0.
I agree with Johan – I am not in a position to argue one way or the other, but I now know enough to pay attention of the direction of the conversation and change/manage my trading behavior accordingly, maybe? I hope?…
Thank you, thank you, thank you, Mark!
This is one of the best blog posts I have read in a long time. IMHO, this post achieves 100% of it’s intended goal/purpose. Very informative and very, very clear (the ‘clear’ part is the one which is most difficult to achieve for any writer/expert).
100% accurate or not, this is a good discussion with some really important questions!
All that brainpower doing nothing more than skimming value from real market innovators…for society, it’s a tragic waste of talent.
I think most of your points are on target. HFT is completely fair, legitimate, and legal. The investor with the fastest information has always had the advantage and this problem will never go away. There will likely always be a price you can pay to get faster information and “beat” o mercado. Nanex has covered a lot of the questionable things that have been done by high-frequency traders. There are many strange things that have occurred that have baffled even industry experts.
I’m not sure how the SEC is going to apply their rules to an algorithm. Math and law don’t really correlate. Back-testing methods that “work” is creating algorithms that always beat the competition. How can the SEC be sure that an algorithm isn’t manipulative or deceptive?
Regardless, I think they are here to stay. In a perfect work, high-frequency traders would comply with all rules and create a market that is so responsive that it’s impossible to cheat or “game” isto. They are the final solution to our real-time markets of the future.
Bela postagem. Question: If HFT causes most market participants to be worse off, why don’t the exchanges regulate it? Note that this could be limited to not providing the asymmetric advantages that you indicate accrue artificially to HFT.

Mais um passo.
Complete a verificação de segurança para acessar a velocidade da luz.
Por que eu tenho que completar um CAPTCHA?
Concluir o CAPTCHA prova que você é humano e dá acesso temporário à propriedade da web.
O que posso fazer para evitar isso no futuro?
Se você estiver em uma conexão pessoal, como em casa, você pode executar uma verificação antivírus em seu dispositivo para se certificar de que não está infectado com malware.
Se você estiver em um escritório ou rede compartilhada, você pode pedir ao administrador da rede para executar uma varredura na rede procurando dispositivos mal configurados ou infectados.
Outra maneira de evitar esta página no futuro é usar o Passo de Privacidade. Confira a extensão do navegador na Loja de Complementos do Firefox.
Cloudflare Ray ID: 3e5c7d47d9bb8b52 • Seu IP: 78.109.24.111 & bull; Performance & amp; segurança por Cloudflare.

How to create a high frequency trading system

Obter através da App Store Leia esta publicação em nosso aplicativo!
Can individuals day-trade stocks using High-Frequency Trading (HFT)?
It seems that all the big trading firms out there use High-Frequency Trading (HFT) to make their transactions and maximize returns from portfolio investments. I was wondering if it is possible for individuals to set up their own HFT system, perhaps on a personal server or something?
The answer is to your question is somewhat complicated. You will be unable to compete with the firms traditionally associated with High Frequency Trading in any of their strategies. Most of these strategies which involve marketing making, latency arbitrage, and rebate collection. The amount of engineering required to build the infrastructure required to run this at scale makes it something which can only be undertaken by a team of highly skilled engineers. Indeed, the advantage of firms competing in this space such as TradeBot, TradeWorx, and Getco comes from this infrastructure as most of the strategies that are developed are necessarily simple due to the latency requirements.
Now if you expand the definition of HFT to include all computerized automated trading you most certainly can build strategies that are profitable. It is not something that you probably want to tackle on your own but I know of a couple of people that did go it alone successfully for a couple of years before joining an established firm to run a book for them.
In order to be successful you will most likely need to develop a unique strategies. The good news is because that you are trying to deploy a very tiny amount of capital you can engage in trades that larger firms would not because the strategies cannot hold enough capital relative to the firms capital base.
I am the co-founder of a small trading firm that successfully trades the US Equities and Equity Derivatives markets. A couple of things to note is that if you want to do this you should consider building a real business. Having some more smart brains around you will help. You don't need exchange colocation for all strategies. Many firms, including ours, colocate in a data center that simply has proximity to the exchanges data centers. You will need to keep things simple to be effective.
Don't except all the group think that this is impossible. It is possible although as a single individual it will be more difficult. It will require long, long hours as you climb the algorithmic trading learning curve. Boa sorte.
Nobody is going to stop you if you want to try that. But you should keep in mind that you have to invest a lot in getting the best hardware you can lay your hands on, best fail-safe connectivity to the exchanges, best trading algorithms and software that money can buy and loads of other stuff. This all needs quite a big amount of upfront investment without guaranteeing returns. That is why you see institutions with deep pockets i. e. banks and trading firms only involve themselves in HFT.
Yes you can, but to do so successfully, you need lots of money. You also need to be able to meet the criteria for being classified as a "professional trader" by the IRS. (If not, you'll be buried in paperwork.)
The fact that you're asking about it here probably means that you do not have enough money to succeed at HFT.
I just finished a high frequency trading project. Individuals can do it, but you need a lot of capital.
You can get a managed server in Times Square for $1500/month, giving you access to 90% of the US exchanges that matter, their data farms are within 3 milliseconds of distance (latency). You can also get more servers in the same building as the exchanges, if you know where to look ;)