12. 12. 2024
Log Management, Log-SIEM
21. 12. 2022
Damiano Chini
Log Management, Log-SIEM, NetEye
How We Verify the Integrity of El Proxy Blockchains Altered by a Retention
El Proxy helps in compliance with GDPR regulations, which, besides the rest, imposes guarantees on the integrity of data and ensures that the data is kept for no longer than a predefined period of time.
El Proxy ensures the integrity of the data by saving the data in El Proxy blockchains. The guarantee that data is kept for a predefined retention period of time is assured by El Proxy via the Elastic index lifecycle management (ILM) policies, which periodically delete the data from the El Proxy blockchains if they are older than the predefined period.
Problem
But how can El Proxy verify the integrity of blockchains if they are modified by the application of ILM policies?
The verification of “full” El Proxy blockchains (not modified by any retention period) starts by verifying from the first block of the blockchain, which is known since it always has iteration 0, and continues from there to verify the entire blockchain.
But what if the first block in the blockchain has been deleted due to the retention period? In this case, El Proxy needs to somehow assess with certainty from which block it needs to verify the blockchain, i.e. it needs to know which is the first block that is expected to be present within the retention period.
If we simply verify from the first iteration currently present in the blockchain, then an attacker who wants to hide some data could simply remove all the data at the beginning of the blockchain and El Proxy would not complain about the blockchain’s integrity. Of course this shouldn’t happen, so we (the NetEye R&D Team) came up with the solution explained below.
Solution
To solve the problem described above, the R&D Team introduced an additional layer into El Proxy verification, which is in charge of determining which is the first iteration (we will call this the “first expected iteration”) that should be verified for a given blockchain. Then the actual verification will start from that iteration.
Determining the “first expected iteration” of the blockchain needs to be based on three parameters:
- The historical data of the blockchain (also called “Blockchain State History”) – in particular this information tells us which iterations are stored in the blockchain for each calendar day – which we’ll talk about this later in this section
- The retention period of the blockchain
- The current date (the date on which the verification is performed)
Assuming we have these parameters, we can assess with certainty which is the first expected iteration because we just need to compute which is the first day within the retention period (i.e. the current date minus the retention period) and then extract from the historical data of the blockchain which is the first iteration in this date. This would be the “first expected iteration”.
While the retention period of the blockchain and the current date can be easily determined during the verification, we need to have access to the historical data of the blockchain.
The solution we designed to achieve this is that the El Proxy verification, besides verifying the blockchain, when successful saves a map to the filesystem containing as keys the calendar days, and as values the first iteration present in that day (for technical reasons we also save the last iteration present in that day).
Subsequent verifications will be able to read this map from the filesystem to determine the “first expected iteration”.
By now you will be wondering what happens in the case where we are performing the first verification, but the Blockchain State History is not yet present. In most cases this is a non-problem because the first verification is usually performed before the retention period deletes the absolute first iteration of the blockchain (the iteration 0), so in this case El Proxy verification can assess with certainty the integrity of the blockchain since it starts from the absolute beginning. If instead the first verification is performed when the iteration 0 has already been deleted, then the first verification will automatically trust the first iteration found in the blockchain and will warn the user about this. All subsequent verifications instead will be safely based on the Blockchain State History saved by the first verification to determine the first expected iteration.
Conclusion
The problem of verifying a blockchain which actually needs to be modified (by ILM policies) after their creation may seem infeasible at first. The key point here is that we know when and how ILM policies modify the blockchains, and so at any given moment we know what the blockchain must contain. We can take advantage of this in the way we explained in the previous section, thus guaranteeing the integrity of the data even when the blockchain is altered by a retention policy.
These Solutions are Engineered by Humans
Did you find this article interesting? Are you an “under the hood” kind of person? We’re really big on automation and we’re always looking for people in a similar vein to fill roles just like this and other roles here at Würth Phoenix.
Damiano Chini
Author
