Segment reordering or OOO packets arriving at TCP has been an old and interesting problem. Main issue is to differentiate between actual loss and reordering so that we can react appropriately.
There are 2 general ways to handle this problem:
1) prevent spurious retransmissions
and/or
2) detect spurious retransmissions as soon as possible and undo the harm done by reduced cwnd and alike.
I could find following relevant work which tries to follow 1):
RR-TCP: Reordering Robust TCP
http://www0.cs.ucl.ac.uk/staff/B.Karp/rrtcp-icnp2003.pdf
TCP-NCR Improving the Robustness of TCP to Non-Congestion Events
https://tools.ietf.org/html/rfc4653
RR-TCP attempt to prevent segment reordering from triggering spurious retransmits by using various algorithms to approximate the duplicate ACK threshold required to disambiguate loss and reordering over a given network path at a given time.
TCP-NCR tried to do the same with a simplified approach as in it simply delays retransmission by an amount based on the current cwnd (in comparison to standard TCP), while the other schemes use relatively complex algorithms in an attempt to derive a more precise value for DupThresh that depends on the current patterns of packet reordering.
Important bit is following in TCP-NCR rfc: "While TCP-NCR offers simplicity, the other schemes may offer more precision such that applications would not be forced to wait as long for their retransmissions. Future work could be undertaken to achieve robustness without needless delay.
In addition, using TCP-NCR could delay the delivery of data to the application by up to one RTT because the fast retransmission point is delayed by roughly one RTT in TCP-NCR. Applications that are sensitive to such delays should turn off the TCP-NCR option."
I've yet to look at how exactly RR-TCP works.
Now, there are other solutions using method 2)
rfc3708: Using TCP Duplicate Selective Acknowledgement (DSACKs)
rfc3522: The Eifel Detection Algorithm for TCP
rfc4015: The Eifel Response Algorithm for TCP
rfc5682: Forward RTO-Recovery (F-RTO)
rfc5682 gets triggered when retransmission timer (RTO) is fired in error. It does not attempt to handle a scenario of spurious retransmission solely based on fast retransmit.
In FreeBSD, we do have some form of "detect RTO caused spurious retransmission" that we handle with CC_RTO_ERR.
rfc3522 on the other hand seems really useful. On top of RTO triggered spurious retransmissions, it also tried to detect fast-retransmit triggered ones. This can be a very good/useful thing to have.
Need to still look into the response part of it i.e. rfc4015 and rfc3708.