The network latency as determined by the driver is what is used to calculate the ‘distance’
I’d suggest looking at the heuristics/rules used for the read preferences Server Selection Algorithm — MongoDB Manual as these may help understand how this is calculated.
The latency window is calculated by the Driver using lowest average round-trip-time on the network between members. This latency window which is the average round trip time of the closest server plus the the
localThresholdMS  gives the window of latency used to select the nearest members.
This process is used for the read preference nearest but it is calculated on a continual basis as part of the background maintenance tasks related to replication between members.
Hope this helps clarify your question.