httpdl: fix progress-reporter race on name/out; harden mirror & resume validation
Findings from a Rob-Pike-lens review (bugs/races/network), each verified against the code before fixing: - httpdl: name/out are now atomic.Pointer[string] -- the engine publishes a download to the reporter before Run resolves the name, so Stat raced the write (bt already did this) - httpdl: a malformed --proxy fails loudly instead of silently bypassing it - httpdl: a 206 must carry a matching Content-Range; a 200 in segmented mode is fatal so it fails over instead of burning the retry budget - httpdl: single-stream mirror failover validates the range before appending; ErrTooSlow only when the error is a real ctx cancellation - httpdl: idle guard tracks progress by timestamp (no Reset/Stop race, no sticky fired flag) - httpdl/control: reject a resume file whose segments don't tile [0,total) - bt: clamp the listen-port range; verify on-disk data before choosing pieces under --check-integrity - cli: reject size overflow; show --seed-time=MIN; clamp --select-file range - progress/engine/main: clamp ETA against int64 overflow; show queued downloads as waiting; join the reporter on exit instead of a 20ms sleep
This commit is contained in:
@@ -4,6 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
@@ -75,7 +76,35 @@ func loadControl(out, url string, total int64, etag, lastmod string) *control {
|
||||
if (lastmod != "" || c.LastModified != "") && c.LastModified != lastmod {
|
||||
return nil
|
||||
}
|
||||
// Reject a control file whose segments do not exactly tile [0,total): a
|
||||
// truncated/corrupted/hand-edited sidecar that still parses as JSON could
|
||||
// otherwise mark a segment done() without its bytes on disk (inflated Written)
|
||||
// or leave an un-downloaded hole, both of which would be reported as a complete
|
||||
// file. We restart cleanly instead of trusting it.
|
||||
if !validSegs(c.Segs, c.Total) {
|
||||
return nil
|
||||
}
|
||||
return &c
|
||||
}
|
||||
|
||||
// validSegs reports whether segs cover [0,total) with no gap or overlap and a
|
||||
// sane written count for each. The recorded order is not sorted (a steal appends
|
||||
// a tail), so we check coverage on a sorted copy.
|
||||
func validSegs(segs []segState, total int64) bool {
|
||||
if total <= 0 || len(segs) == 0 {
|
||||
return false
|
||||
}
|
||||
sorted := append([]segState(nil), segs...)
|
||||
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Start < sorted[j].Start })
|
||||
var next int64
|
||||
for _, s := range sorted {
|
||||
length := s.End - s.Start + 1
|
||||
if s.Start != next || s.End < s.Start || s.Written < 0 || s.Written > length {
|
||||
return false
|
||||
}
|
||||
next = s.End + 1
|
||||
}
|
||||
return next == total
|
||||
}
|
||||
|
||||
func removeControl(out string) { os.Remove(controlPath(out)) }
|
||||
|
||||
Reference in New Issue
Block a user