Immediate and accurate analysis of financial time series data is crucial to the price discovery mechanism that is at the heart of capital markets.
We’ll show you how insights can be derived from financial time series data, in real-time, using Machine Learning. In particular, a Keras model implementing an LSTM neural network for anomaly detection is provided.
The data we’re using comes from the Band Protocol public dataset available in Google BigQuery. Band Protocol is a cross-chain data oracle platform that aggregates and connects real-world data and APIs to smart contracts. …
In this article we’ll cover our Ethereum 2.0 ETL tools for exporting Ethereum 2.0 blockchain data, the public Medalla dataset in BigQuery, and a Nansen.ai dashboard we built with some interesting charts and tables.
The article is broken down into three parts:
Now, let’s go through the details.
bigquery-public-data.crypto_ethereum dataset in BigQuery.
The whole process can be broken down into three steps:
webpack.config.jsand build a JS file using webpack.
This is a tutorial article explaining how to replay time series data from a BigQuery table into a Pub/Sub topic. There are several use cases when you might need it:
The go-to GCP service for moving data between different services is Dataflow. While there are many Google-provided Dataflow templates, there are none for moving data from BigQuery to Pub/Sub.
That’s why we developed our own tool to solve this task: https://github.com/blockchain-etl/bigquery-to-pubsub. It can be used to replay any BigQuery table with a TIMESTAMP field. It’s a Python program that sequentially pulls chunks of data…
blockchain-etldatasets. Find instructions below.
You can select each table and view its schema, details, and preview the data on the bottom right.
Try pasting this…
The overall architecture is depicted below:
The following blockchains are covered:
We added delays for each blockchain that prevent streaming orphaned blocks resulting from chain reorganisations. You can look up how many blocks we lag behind the tip of the chain in the
LAG_BLOCKS parameter in the configuration files in the Github repository https://github.com/blockchain-etl-streaming. Those values were calculated based on the longest orphaned chains within the last year…
For every law the author provides an example from history, interprets and explains it and gives advice on how to use the law.
With our recent release of Bitcoin-derived blockchain datasets, BigQuery now contains 8 cryptocurrencies in total including Bitcoin, Bitcoin Cash, Zcash, Litecoin, Dogecoin, Dash, Ethereum, and Ethereum Classic. Below is the graph demonstrating daily transaction counts for those blockchains:
Ethereum is clearly leading with almost 600k daily transactions on average in Jan 2019. The highest daily transactions were seen in Jan 2018 with almost 1.2M tx/day, which is around 14 tx/sec on average.
You can also see the Ethereum DAO and the Bitcoin Cash forks on the top graph. …
In this article I will guide you through the process of creating an ERC20 token recommendation system built with TensorFlow, Cloud Machine Learning Engine, Cloud Endpoints, and App Engine. The solution is based on the tutorial article by Google. The data used for training the recommendation system is taken from our public Ethereum dataset in BigQuery.
The article is broken down into the following parts:
The Gini coefficient, also known as the Gini index, is a common econometric tool for measuring inequality of asset distribution.
Here is the query that outputs Gini coefficient for each day given daily non-zero (anonymous) account balances:
1 — 2B formula from this Wikipedia page https://en.wikipedia.org/wiki/Gini_coefficient, where B is the area under the Lorenz curve:
balance * (rank — 1)is the area of the rectangular horizontal slice under the Lorenz curve.
balance / 2is the area of the triangle on the left of the rectangular slice.
sum((balance * (rank —…