Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
6.170536894s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:11:27 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.170562223s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.284007834s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:10:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.284039825s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.756727657s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:09:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.756749338s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.220168082s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:08:53 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.220190004s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
6.679458592s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:07:50 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.679485804s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.160865208s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:07:34 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.160890126s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.217588225s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:06:54 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.217634493s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.709022958s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:05:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.709053526s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.165543331s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:05:06 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.165566585s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.703480629s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:03:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.703506548s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.671669252s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:03:04 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.671695012s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
6.56186734s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:02:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.561891808s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.633662517s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:01:40 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.633687184s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.945720993s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 01:00:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.945749347s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.234392769s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:59:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.23443538s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
7.219716764s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:59:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.219740108s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.552677402s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:58:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.552699745s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.491495371s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:58:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.491520709s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
6.966669039s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:57:31 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.96669558s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.547978438s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:57:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.548000321s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.85919213s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:56:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.859215435s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.749778448s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:56:37 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.749802585s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.977875572s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:56:08 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.977943372s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [X...X.......................................................] duration:
6.763405145s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:55:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.763428308s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.715998978s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:55:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.716022785s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.X...X......................................................] duration:
6.655230866s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:54:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.655286501s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.72303385s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:54:33 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.723057033s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration:
6.837213467s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:54:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.837238164s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.727333296s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:53:46 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.727363394s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X...X....................................................] duration:
6.753037269s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:53:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.753062188s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
6.614564375s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:53:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.614586647s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
6.594999153s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:53:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.595023509s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....X...X...................................................] duration:
6.561885632s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:52:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.561905489s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.648319436s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:51:58 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.648340876s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration:
7.053711948s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:51:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.053733911s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
6.231665084s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:51:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.231685091s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......X...X.................................................] duration:
6.906622348s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:50:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.90665027s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.83324235s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:50:24 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.83326887s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......X...X................................................] duration:
14.100759261s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:50:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 14.100844815s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.320452455s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:49:37 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.320479688s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
6.540090145s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:49:17 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.540116564s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........X...X...............................................] duration:
18.164248917s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:49:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.164409824s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.860737545s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:48:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.860762232s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........X...X..............................................] duration:
18.553444477s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:48:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.553467121s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
6.169640071s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:48:02 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 6.169668746s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.796579938s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:47:41 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.796601949s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........X...X.............................................] duration:
43.602414616s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:47:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 43.602438974s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........X...X............................................] duration:
29.063842749s trace:
https://jaeger.dev.storj.io/trace/7667f23ab5b366fa
FAIL (2026-05-06 00:46:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.063866616s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.585304164s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:46:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.585318511s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.782609267s trace:
https://jaeger.dev.storj.io/trace/7309bace5fda504b
FAIL (2026-05-06 00:45:45 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context canceled
======LOGS======
.374554348 (125h49m35.895378093s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 6 00:37:25.611723918 (125h49m36.132547663s) [seg_pos={0 0}, upload=120443]: Begin upload segment...
May 6 00:37:25.611788120 (125h49m36.132611885s) [seg_pos={0 0}, upload=120443]: Done begin upload segment.
May 6 00:37:25.611836733 (125h49m36.132660478s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.611855860 (125h49m36.132679615s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.612527341 (125h49m36.133351086s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613061881 (125h49m36.133885636s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.613512341 (125h49m36.134336096s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.614707422 (125h49m36.135531177s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.615796219 (125h49m36.136619964s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.616279842 (125h49m36.137103597s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.617598358 (125h49m36.138422103s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.618925220 (125h49m36.139748965s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620260308 (125h49m36.141084053s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.620633770 (125h49m36.141457515s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.621928260 (125h49m36.142752005s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.622848366 (125h49m36.143688983s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.623736612 (125h49m36.144560357s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:25.625856647 (125h49m36.146680392s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Uploading piece...
May 6 00:37:27.322013072 (125h49m37.842836827s) [upload=120443]: Next returned nil segment
May 6 00:37:27.322025576 (125h49m37.842849321s) [upload=120443]: Waiting for error group managing segments...
May 6 00:37:27.604612373 (125h49m38.125436118s) [noise=true , address=145.79.227.40:4050 , piece_id=5XDNNSC66QJ4PIOZ5WA57UCW3BWHLCPRMUY3UQCSVZ72P62DYWEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615428994 (125h49m38.136252739s) [noise=true , address=145.79.227.17:4971 , piece_id=UJ34ZA5E3UM6CRQ67FKZ65XR3BOXR27BNAYPHPV4TYIFFE3VO5KQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.615840539 (125h49m38.136664284s) [noise=true , address=145.79.227.24:4584 , piece_id=6TPNOGPN4HDKF25CNDHMZWKNWDNAEJL5NQBOA4DZJDWW77I4LDQA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.616063335 (125h49m38.13688708s) [noise=true , address=145.79.227.36:4521 , piece_id=HNC3ZXXAOCPRFUYFWB36N4I52HUZBLW57X2P22UVC3H6D4WXPA4Q, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.624558967 (125h49m38.145382712s) [noise=true , address=145.79.227.13:4806 , piece_id=LYT7BTXABBNU3VU7ATSO5CZI5KCHCTRR3TJFHM6LCVCYXILVZU6A, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.630019938 (125h49m38.150843683s) [noise=true , address=145.79.227.41:4085 , piece_id=KKBW4XLFMN2MNP2WEA6ETDRXS4BI5ZQRAM2LETVHPYKUYR4XSSVA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631393028 (125h49m38.152216783s) [noise=true , address=145.79.227.8:4335 , piece_id=WFWAGOAFWVIYA3ED4XPM2XUV7O6RO4TBNGNIYSZSHX2Y4YMBL2RA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.631619800 (125h49m38.152443546s) [noise=true , address=145.79.227.28:4726 , piece_id=CI3WYWDOEUTHB3LM5GNWOEWLE3SYQLVGRJXF5H2O5Q6PPIPTFTEQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.637472841 (125h49m38.158296576s) [noise=true , address=145.79.227.6:4259 , piece_id=55PTWNU7EIGOXD3ZFOCP4LL5CUXLVXBHUTCG3TVCU5ZIDQDHU4OA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.646377433 (125h49m38.167201178s) [noise=true , address=145.79.227.21:4474 , piece_id=3ULB5FMJH2ILASZ4Q5LLZPB3WOHZQ2PU2BSMFPPXFH47KNK7Q4VA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.653836798 (125h49m38.174660543s) [noise=true , address=77.90.31.10:4521 , piece_id=IQPFDPZZNULZRLZAOBMHWQJ7A6KNH7FDY55QKKB5P4P2OFW7I6WQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.655850541 (125h49m38.176674286s) [noise=true , address=145.79.227.11:4735 , piece_id=DEFZDSDDDJABXMCMSEAE3SOUSD2S4EL6E2EOEGYG5E6RTYCXUFFQ, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677700066 (125h49m38.198523811s) [noise=true , address=145.79.227.18:4010 , piece_id=JY5RL4EW2TOY5V2UVDMUUAJ7GI34P4NDC7FMT5QTUH6B3PBSPNKA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: <nil>
May 6 00:37:27.677718030 (125h49m38.198541775s) [seg_pos={0 0}, upload=120443]: Satisfied cohort requirements after 13 pieces
May 6 00:37:27.677917330 (125h49m38.198741076s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N1YuMnuTFjGboQrrfxXEbJm4HwhADdNn8F5c7wWwQYpjgFZoY): protocol: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49622->45.91.116.4:4941: use of closed network connection
May 6 00:37:27.677923412 (125h49m38.198747157s) [noise=true , address=45.91.116.4:4941 , piece_id=4S76Q7JW7LSKPTRNHSJ4LH7HHI7L7DUM44J5UJEYHLT6DSDFEFDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678018343 (125h49m38.198842088s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55782->45.91.116.13:4156: use of closed network connection
May 6 00:37:27.678024023 (125h49m38.198847759s) [noise=true , address=45.91.116.13:4156 , piece_id=OLDZUHLDZJTH5U5QPUOUDQ3UKNUQMFFZ7CMHX6VWTAYX5MZQCYEA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678031679 (125h49m38.198855424s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51032->45.91.116.7:4009: use of closed network connection
May 6 00:37:27.678048490 (125h49m38.198872235s) [noise=true , address=45.91.116.7:4009 , piece_id=KERWC67HPMMDUY3RXHWQLHBK3C36MFFPX2ZIXRFH652AZGZEAMDA, seg_pos={0 0}, upload=120443]: Long tail cancellation due to optimal threshold
May 6 00:37:27.678113885 (125h49m38.19893763s) [seg_pos={0 0}, upload=120443]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 6 00:37:27.678193537 (125h49m38.199017322s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 6 00:37:27.678198717 (125h49m38.199022462s) [upload=120443]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 6 00:37:27.970416542 (125h49m38.491240277s) [upload=120443]: Done uploading segments...
May 6 00:37:27.970433054 (125h49m38.491256799s) [upload=120443]: Upload finished. err: <nil>
Duration: 5.782638642s
core
uplink
us-select-1
|