EIP-8: devp2p Forward Compatibility Requirements for Homestead - #49

In my opinion the proposal is solid apart from the RLP encoding question I've raised above. That should be looked at by someone more capable in cryptography.

I would however like to emphasize that this would be a very high priority feature to get into the Homestead fork, as the protocol in it's current state is not upgradable without a hard fork. There are issues with the current protocol already, and as time progresses I'm certain we'll find plenty more, or simply much needed extensions that cannot be implemented due to the forking requirements.

The most important aspect IMHO about the solution proposed by Felix is that it's extremely simple. It could be added to any implementation in a matter of minutes, which is imho worth doing now and paving the way forward to easier upgrades afterwards.

The upgrade mechanism itself is quite crude as any version bump requires adding fields to the end, no fields can be deleted or semantically changed (to keep backwards compatibility), but as we're not aiming to do many updates, just to facilitate occasional essential ones, I think this is fine. And as we're transitioning through metropolis and serenity, we can take the lessons learned from previous updates and refine/clean in each of these hard forks.

Best case scenario we can seamlessly upgrade the devp2p protocol as need arises, worst case scenario we don't use the upgrade mechanism, but it won't hinder anything either.

Currently the id's are 64 bytes. Can we change them 32 bytes (a compressed public key)?

@karalabe If homestead is a hard-fork I think we should just fix the protocol. A design rationale behind RLPx is that communication MUST NOT rely on the authentication handshake for metadata, and that metadata is to be exchanged via discovery. The solution is so simple: Try to connect to the node via discovery first! This way both sides will authenticate what version they know they can speak before attempting the handshake – its also easy to write a discovery (UDP) stack which supports multiple versions. Case in point, a peer cannot connect to a node prior to discovery – as discovery is how the public key is obtained. No discovery == no TCP handshake.

@karalabe @fjl @wanderer

For the handshake, what do we need to consider other than the scenarios below?

Node AAA, Version 1
Node FFF, Version 2

Scenario 1: Node AAA(v1) attempts connecting to FFF(v2)
Scenario 2: Node FFF(v2) attempts connecting to AAA(v1)

Note: This upgrade is more complex than it seems. The necessity of this upgrade is in order to fix the way the KDF used and to enable framing. Ideally we also remove AES padding. What this means is that if new clients are to support old clients they will need to be able to enable the new framing system on a per-session.

@subtly I'm not planning on supporting backwards compatibility.

And that's fine. You can always drop connections with lower version (the EIP doc explicitly permits this).

Test vectors have been added.

@subtly (aggregating our discussion a bit)

Your concerns about the RLPx handshake changes seem to be:

• Adding plaintext of any form is bad. I agree that plaintext that leaks secrets is bad, but the size of the handshake is not a secret.
• The size prefix makes identifying RLPx TCP connections easy. My take on it is that the fixed size handshake is just as recognizable and that randomizing the length offers adequate protection in most cases.
• The handshake should not carry metadata, it should be communicated through discovery instead. The issue with this is that discovery isn't always available (e.g. when connecting through Tor). We are also planning to offer discovery over RLPx/TCP in the near future, in which case the RLPx handshake is the entry point.
• It requires a backwards-compatible implementation for the RLPx v5 KDF changes. If you don't care about backwards-compatibility, you can refuse communication with older nodes. The change merely shifts responsibility so the newer node is in control about this decision.

After discussing with @fjl and @nagydani I think we need to write up a spec of exact requirements of RLPx in terms of threat models - what attacks are we interested in defending against right now, and which can we leave for future protocol upgrades. It's hard to have a discussion of details such as leaking plaintext and/or ciphertext length without having consensus on threat models & general reqs.

For anyone following this discussion: A lot has happened. We've had multiple long calls about EIP-8, a counterproposal and a rebuttal of the counterproposal have been written.

I've updated the text of EIP-8 to clarify some things. Notably, Postel's Law is now mentioned because it
might not be so obvious that it is at play here. A new Rationale section on the discovery change has been added, and there is now a section exploring the implications of the plaintext size prefix for the handshake packets.

However, such an attacker can also be expected to read and or participate in RLPx Discovery traffic, in which case effective blocking is possible irrespective of the size prefix.

So I think in that case the discovery traffic wouldn't be UDP. It could be by written notes sent by pigeons, ect. But the point is just because the normal method for discovery is blocked the transport can still operate.

The document assumes nodes talking using the currently specified protocols, not a version where arbitrary transport is negotiated using an arbitrary method of discovery. Not sure whether this needs to be made explicit. We can improve on everything in the future, but for the RLPx protocol that is in use right now, correlating UDP traffic is definitely possible (and there are other indicators, including commonly used ports, the fixed-size handshake traffic pattern, the first byte always being 0x04, etc.).
I guess what I'm trying to say is that the prefix won't make things worse than they are now.

The document assumes nodes talking using the currently specified protocols, not a version where arbitrary transport is negotiated using an arbitrary method of discovery

I think I have a slightly different view here. I view the three protocols (discovery, handshake, transport) as being mutually independent and composable.

This is my abstract view, too, but when discussing the possibility of filtering RLPx v4 TCP connections
we need to look at what people actually use. There are two choices for users now: using UDP discovery or supplying the pubkey+endpoint. There won't be a third option just tomorrow, unless we implement it (or use a library for it ;)) and it gets adopted.

Anyway, I've fixed up the sentence to state explicitly that TCP connections could be blocked by looking at UDP traffic, which was the intended meaning.

Another short overview document (as requested by @chriseth): https://gist.github.com/6c6d24dc1c7ab5d77c5c

Note: test vectors need to be regenerated for the size prefix authentication change. I'll do that and then squash all the commits together.

Test vectors are now up to date.

Squashed.

Could we get this merged into the EIP repo if it was accepted for homestead? (Also specify it as "Accepted"?)

@karalabe yes definitely! @fjl could change the status to "accepted" on this? It would b ready to merge then

Done.