Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.063298496s
FAIL (2025-10-07 10:51:35 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.063329255s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.836127466s
FAIL (2025-10-07 10:51:27 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.836157213s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
6.457837288s
FAIL (2025-10-07 10:51:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.457863958s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
6.511536721s
FAIL (2025-10-07 10:51:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.511568111s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.15337925s
FAIL (2025-10-07 10:50:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.153405609s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
6.034716492s
FAIL (2025-10-07 10:50:43 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.034753584s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.109226302s
FAIL (2025-10-07 10:50:43 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.109262041s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.215417068s
FAIL (2025-10-07 10:50:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.215445392s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.157750771s
FAIL (2025-10-07 10:50:19 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 6.15779169s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
6.371529712s
FAIL (2025-10-07 10:50:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.371562936s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.215463439s
FAIL (2025-10-07 10:49:40 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.215492483s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.269493337s
FAIL (2025-10-07 10:49:38 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.269575202s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.44789628s
FAIL (2025-10-07 10:49:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.44795489s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.861244997s
FAIL (2025-10-07 10:49:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.861285543s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
6.137424819s
FAIL (2025-10-07 10:49:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.137459434s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
6.11422951s
FAIL (2025-10-07 10:48:36 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.114251612s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.973645323s
FAIL (2025-10-07 10:48:35 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.973673036s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.090166738s
FAIL (2025-10-07 10:48:27 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.090195182s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.110036175s
FAIL (2025-10-07 10:48:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.110063087s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.840857613s
FAIL (2025-10-07 10:48:01 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.840887558s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.952335943s
FAIL (2025-10-07 10:47:35 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 1.952366022s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.964221578s
FAIL (2025-10-07 10:47:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 5.964249762s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.802979365s
FAIL (2025-10-07 10:47:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.803010494s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.334515724s
FAIL (2025-10-07 10:47:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.334534629s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
6.001077529s
FAIL (2025-10-07 10:46:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.00111039s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.044643355s
FAIL (2025-10-07 10:46:25 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.044683221s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.976419517s
FAIL (2025-10-07 10:46:14 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.976452848s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.187253039s
FAIL (2025-10-07 10:46:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.187281023s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
6.099433205s
FAIL (2025-10-07 10:45:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.099454806s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.092888063s
FAIL (2025-10-07 10:45:47 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.092917338s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.780421622s
FAIL (2025-10-07 10:45:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.780457289s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.40393048s
FAIL (2025-10-07 10:45:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.403952814s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
6.084165955s
FAIL (2025-10-07 10:45:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.084195822s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.989495496s
FAIL (2025-10-07 10:45:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.989532044s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.072966177s
FAIL (2025-10-07 10:45:21 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.073053563s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.435809222s
FAIL (2025-10-07 10:45:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.435830873s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.053516232s
FAIL (2025-10-07 10:45:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.053558594s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.98143229s
FAIL (2025-10-07 10:44:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 5.981467798s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.133608282s
FAIL (2025-10-07 10:44:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.133658748s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.678054648s
FAIL (2025-10-07 10:44:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.678100184s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.075459965s
FAIL (2025-10-07 10:44:09 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.075493929s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.039088217s
FAIL (2025-10-07 10:44:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.039122903s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.899947781s
FAIL (2025-10-07 10:43:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.89997916s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.159851349s
FAIL (2025-10-07 10:43:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.159879562s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
6.410820359s
FAIL (2025-10-07 10:43:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 7 10:12:08.558107699 (332h40m54.020200182s) [upload=292768]: Starting upload Oct 7 10:12:08.558204974 (332h40m54.020297457s) [upload=292768]: Done starting upload Oct 7 10:12:08.558299444 (332h40m54.020391927s) [upload=292768]: Uploading segments... Oct 7 10:12:08.558492031 (332h40m54.020584514s) [upload=292768]: Got next segment. Inline: false Oct 7 10:12:08.558535513 (332h40m54.020627996s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Oct 7 10:12:08.558544311 (332h40m54.020636804s) [upload=292768]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Oct 7 10:12:37.855472289 (332h41m23.317564762s) [upload=292768]: Next segment err: context deadline exceeded Oct 7 10:12:37.855578451 (332h41m23.317670924s) [upload=292768]: Done uploading segments... Oct 7 10:12:37.855594531 (332h41m23.317687014s) [upload=292768]: Upload finished. err: context deadline exceeded
Duration: 6.410853123s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.108818135s
FAIL (2025-10-07 10:43:09 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 6.108848504s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.101710654s
FAIL (2025-10-07 10:43:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.101737936s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.109956414s
FAIL (2025-10-07 10:43:01 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 2.109989085s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.889968678s
FAIL (2025-10-07 10:42:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.889999747s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.941627557s
FAIL (2025-10-07 10:42:13 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
2:42.165264348 (332h41m27.627356861s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.165955152 (332h41m27.628047625s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.166898680 (332h41m27.628991163s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167359617 (332h41m27.62945209s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.167858417 (332h41m27.6299509s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.168663159 (332h41m27.630755642s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169008787 (332h41m27.63110126s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169409100 (332h41m27.631501583s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.169959709 (332h41m27.632052192s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.170441054 (332h41m27.632533537s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171142340 (332h41m27.633234834s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171422123 (332h41m27.633514606s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.171936802 (332h41m27.634029286s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172419642 (332h41m27.634512165s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.172505987 (332h41m27.63459847s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173003063 (332h41m27.635095546s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:42.173354913 (332h41m27.635447396s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Uploading piece...
Oct 7 10:12:43.558596988 (332h41m29.020689461s) [upload=292777]: Next returned nil segment
Oct 7 10:12:43.558619280 (332h41m29.020711753s) [upload=292777]: Waiting for error group managing segments...
Oct 7 10:12:43.911075424 (332h41m29.373167907s) [noise=true , address=64.49.13.27:4959 , piece_id=Q7IVCXASNNEVEJF6PSNQOFMX4GOR4VQNWXI4K67H4T3WJRGRERVA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:43.999387778 (332h41m29.461480251s) [noise=true , address=145.79.227.23:4575 , piece_id=RBUQMX6L7HRE7777AIAMNUEIRHR6AFC33DMVQANBKEGUX5DVXEBA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.008759212 (332h41m29.470851685s) [noise=true , address=145.79.227.42:4138 , piece_id=B2EG7P62T6Z7MV3WAKFRGYH3WQNNHDWYQL5ONKYJRN7BIBCVI2CA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010009934 (332h41m29.472102417s) [noise=true , address=145.79.227.19:4106 , piece_id=5W2IHZIDRVT4N65DW5EKD54QQSQAU5CC4IAENWMQIIMSMMBJIAOQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.010760634 (332h41m29.472853117s) [noise=true , address=145.79.227.24:4591 , piece_id=XUGNOWN33T62WB5L7CPI4BDNIJHHYNWKCW3RFFGVCSX7JQICJNIQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.027043800 (332h41m29.489136283s) [noise=true , address=145.79.227.20:4447 , piece_id=OJMCBKSVDODKQBP63HGNCE2J6CG7ZGJTJHC5ZHY6HO552L3IWVJA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029164559 (332h41m29.491257032s) [noise=true , address=145.79.227.29:4952 , piece_id=N3I32ETGCV5Q7VPWSIH5NBSU3DWHL3B4MVY5AGIL35YUGPIPWVYA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.029210005 (332h41m29.491302488s) [noise=true , address=145.79.227.38:4004 , piece_id=JBNMBUDYP6L4DDUZMLKWC56XR4C7XYVXV7ECBUUTMOIKSHJOZ6AQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.033972373 (332h41m29.496064856s) [noise=true , address=145.79.227.32:4840 , piece_id=2VMNF4ESFOC4FD6XIJXOH2OYW7KOIP5FTWNC43BJOKMRCRM2XP2A, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.034737540 (332h41m29.496830013s) [noise=true , address=145.79.227.52:4458 , piece_id=PWK65JVKZJOKTRJ3RUFWRARU6OMYAQJ45ALDFY47BIHBOWSMA2XA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.142533886 (332h41m29.604626369s) [noise=true , address=77.90.31.15:4730 , piece_id=DNLCZ75DDJ727UKPE4BLQ4RW2F7CONINHHO7RE325D5EWWA5JDTA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.158687164 (332h41m29.620779637s) [noise=true , address=77.90.31.18:4814 , piece_id=VDA7BKMPAMNXMUZXYKLD3ECQRLPTQTFCRCOZSBUXQ34WRUAR6JFA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.174538559 (332h41m29.636631032s) [noise=true , address=45.91.116.5:4312 , piece_id=5P6KE7HIRAEGY2PRHLZSB42LZS6RRBZWUQGGMP3EISRETUT6YR5Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206699178 (332h41m29.668791661s) [noise=true , address=45.91.116.14:4215 , piece_id=45BEDZCAYMQWZJBETXDADIBBG2A6LKYMVXI46HBYVUKR6S6JGJHA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: <nil>
Oct 7 10:12:44.206743001 (332h41m29.668835484s) [seg_pos={0 0}, upload=292777]: Segment reached optimal threshold of 14 pieces.
Oct 7 10:12:44.206951859 (332h41m29.669044342s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.206961288 (332h41m29.669053771s) [noise=true , address=77.90.31.21:4885 , piece_id=QZQ7O2X56Y5Y24LG6OHRHGZMHZJ5ZGB7ACZMCDRPK3U7HSLAUAZA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207002376 (332h41m29.669094859s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207022453 (332h41m29.669114936s) [noise=true , address=45.91.116.8:4064 , piece_id=RB5BLW65DRMAEZ5DZNAOKBTX2CGBIAEJXGREKNJQCEZJEEQYMZ4Q, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207077298 (332h41m29.669169781s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Eki7jYegwvkdixR5pznQBbSeJPn9GjGofxnUApidQF4kRNBK5): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207086465 (332h41m29.669178938s) [noise=true , address=45.91.116.4:4963 , piece_id=D43ZVDFIGGIPIXQMMIHJOSQK6EHS54RTKUYMAK5I4XTB4LNLHKYQ, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207177830 (332h41m29.669270303s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF
Oct 7 10:12:44.207186476 (332h41m29.669278949s) [noise=true , address=77.90.31.19:4860 , piece_id=QRWWQSWR6HLKTJ4R6PREPXYC5QONDP4T6DWNBG5KGT6K6TY2TUPA, seg_pos={0 0}, upload=292777]: Long tail cancellation due to optimal threshold
Oct 7 10:12:44.207212376 (332h41m29.669304859s) [seg_pos={0 0}, upload=292777]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Oct 7 10:12:44.207297427 (332h41m29.66938991s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 7 10:12:44.207302326 (332h41m29.669394799s) [upload=292777]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 7 10:12:52.521007048 (332h41m37.983099531s) [upload=292777]: Done uploading segments...
Oct 7 10:12:52.521031375 (332h41m37.983123848s) [upload=292777]: Upload finished. err: <nil>
Duration: 5.941664336s
core
uplink
us-select-1
us1-placement-20
|