In theprevious part,we paid attention to the process of accepting ETH. In this article, we'll make a similar analysis of dealing with ERC20 tokens.
Let’s outline the task. I need to receiveERC20token payments, (1) link them to a specific purchase, and (2) accumulate in a single wallet. In the off-chain world of bank transfers, this issue is usually addressed in the following way. I register a bank account and give its details to customers along with a unique purpose (which is a universally available, a standard field in a bank transfer details). A sender must specify this information while making the transaction. In this way, (2) is achieved naturally as all the transfers end up in the same place, and (1) is achieved by looking for purpose in the incoming payments.
I cannot directly apply the same architecture to Ethereum because there is no such thing as purpose in the ERC20 standard. What if, instead, I create unique wallets for every purchase? This operation is fast and free and solves (1) naturally. To solve (2), I have to add an extra step to the payment processing and sweep incoming tokens to my main wallet.
After an incoming payment I cannot send a token transaction from a wallet that has 0 ETH on its balance. So, after the payment processing is triggered, I will send enough ETH to the wallet, and after it has arrived, I will make a transaction transferring tokens from the wallet.
How do I know that tokens already hit the unique wallet? What will trigger an actual payment processing? A naïve way to go is by polling the token balance of a unique wallet every time a new block is mined (about 15 seconds) till the moment it is greater than 0 (or some appropriate limit) and then proceed with handling. This approach, though absolutely failsafe, have some scalability issues. What if I have 100,000 pending purchases? It means that I have to do 100,000 token balance checks every 15 seconds, so 6600+ requests/second. After what time of no payment should I stop polling a particular wallet? What if I stop, but the payment arrives later? I can add some heuristics and restart polling based on a user's actions, like checking the purchase status. Or I can trigger the wallet processing manually, in case the user raise a support ticket. No matter which optimizations I apply, it seems that the number of simultaneous polling requests will increase linearly over time, assuming that my service is performing and the number of users/purchases increases.
How can I redo the trigger logic so that it works in real time and doesn’t require users to contact the support? I can inspect every transaction to check out if:
- A receiver is the address of my subject ERC20 Token Contract, AND
- The Transaction Data contains the token receiver address from my list of wallets
Now it doesn’t matter how many pending purchases I have. What matters is how many transactions there is to parse. So, if there are 10 million transactions per day, then I only need to be fast enough to parse 120 transactions per second. And it is doable.
Let’s dive into details on a transaction parser implementation. For the sake of simplicity, I’m not even interested in the amount of incoming transaction. As long as I notice one of my unique wallets in there, I trigger a balance check and do the sweep
There is a To field in every Ethereum transaction. So, I’m just going to check if it matches the ERC20 Token Contract address to satisfy 1. Then I’m going to check if the data is atransfer call and I will extract the receiver from it. Here is a regex that will do the job:
When the receiver is extracted, I’ll check if it is one of my unique wallets to achieve2 and to trigger processing if it does. Now everything is covered, right?
Here I see that To field doesn’t match the ERC20 Token Address, so 1 is not achieved. In addition, the data is empty, so 2 is not achieved either. Nevertheless, the tokens have arrived. How can I trigger processing in such case? Note: the previous approach with the balance polling processes, in this case, is out of the box. The only feasible way to identify such transactions, currently, is by using aParitytrace module’strace_replayTransaction. It will show me all the internal transactions.
And again, there is a To field in every internal transaction, and it does match the ERC20 Token Address, so 1 is achieved. Also, there is an input data field which I can parse as before to achieve 2. I know that the first entry in the trace is the original external transaction, so I now have a single routine (look only into trace) to find all the transfers. The problem is that my handler will only notice transfer calls. It will not notice transferFrom which is also just a token transfer. And it will not notice any custom calls leading to the tokens transfer.
What do all these cases have in common? Somehow Etherscan is able to identify transfers of any ERC20 tokens, no matter how exactly the transfer occurred. Let me recheck theERC20specification again.
There is a thing in Ethereum which is called Events (or logs). Luckily, every token that follows the ERC20 specification guarantees that Transfer event will be triggered every time the tokens are being transferred. This is exactly what I was looking for. How do I get those logs? I need to modify my transaction parser to parse the logs inside of the transaction and ignore everything else.
Now I don’t even need the traces anymore. What I need to do is to go through every transaction, extract the logs array, and in every log check that:
- An address that triggered the event is the ERC20 Token Address;
- topics equals to Transfer event signature 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef
- topics matches one of my unique wallets.
If all these checks are passed successfully, I trigger the payment processing and everything to be done with it. Is there any way to make things even easier to integrate? Can I get a list of all the Transfer events, of all the ERC20 Tokens, and then just iterate over it? Apparently, I can! Assuming that I want to get all Transfer logs from block 4888378 (I can get events from a span of blocks at once!), I need to perform a singlerequestto my Ethereum node:
curl -X POST --data
There are 51 Transfer logs in that block. And it took less than a second for my node to process the request. From now on, I can process any ERC20 payments without the need to update the handler at all, even if new tokens are being added. I don’t care how exactly a transfer occurred as soon as I have an event, I trigger the payment processing.
To my knowledge, a vast majority of the ERC20 integrations only trigger the payment processing for the first situation (e.g., an initial transaction To the ERC20 Token Address, the data is transfer call, and a receiver is their wallet address). They completely or partially ignore all other cases. Some of them will trigger the processing manually through support; others will just take money to themselves, arguing that “Only normal transactions are accepted.” I want to emphasize that all cases described above are normal because they all are the part of ERC20 Standard, and that is how Ethereum works. It isforecastedthat in the future all ERC20 transfers will be of any type but the first (e.g., through an internal transaction).
“... we take initial steps towards a model where, in the long term, all accounts are contracts...”