Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..................X......................................] duration: 14.357552394s FAIL (2025-10-09 16:04:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 14.357590426s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..................X.....................................] duration: 15.532734639s FAIL (2025-10-09 16:04:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 15.532768202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..................X....................................] duration: 6.317845986s FAIL (2025-10-09 16:02:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.317879961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X....X..................X...................................] duration: 19.024783624s FAIL (2025-10-09 16:02:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 19.024821516s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X....X..................X..................................] duration: 8.223973469s FAIL (2025-10-09 16:01:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 8.224074232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X....X..................X.................................] duration: 7.217099297s FAIL (2025-10-09 16:00:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 7.217130697s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X....X..................X................................] duration: 10.518409606s FAIL (2025-10-09 15:59:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.518446938s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X....X..................X...............................] duration: 6.455994453s FAIL (2025-10-09 15:59:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.456027757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X....X..................X..............................] duration: 6.438495476s FAIL (2025-10-09 15:58:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.438527736s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X....X..................X.............................] duration: 9.18318568s FAIL (2025-10-09 15:57:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 9.183215667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X....X..................X............................] duration: 10.580306556s FAIL (2025-10-09 15:57:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.580335431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X....X..................X...........................] duration: 6.599574834s FAIL (2025-10-09 15:56:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.599609049s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X....X..................X..........................] duration: 6.841244078s FAIL (2025-10-09 15:55:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.84127582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X....X..................X.........................] duration: 7.6949726s FAIL (2025-10-09 15:54:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 7.695007106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X....X..................X........................] duration: 8.54790668s FAIL (2025-10-09 15:53:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 8.547939903s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X....X..................X.......................] duration: 9.544901679s FAIL (2025-10-09 15:53:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 9.544937296s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X....X..................X......................] duration: 9.200545827s FAIL (2025-10-09 15:52:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 9.200578659s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X....X..................X.....................] duration: 12.231107895s FAIL (2025-10-09 15:51:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 12.231139386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X....X..................X....................] duration: 6.540203137s FAIL (2025-10-09 15:50:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.540235579s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X....X..................X...................] duration: 6.662291922s FAIL (2025-10-09 15:50:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.662317831s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X....X..................X..................] duration: 7.443833323s FAIL (2025-10-09 15:49:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 7.443868009s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.................X....X..................X.................] duration: 11.366553085s FAIL (2025-10-09 15:48:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 11.366580899s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.................X....X..................X................] duration: 6.636765931s FAIL (2025-10-09 15:47:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.636795045s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.................X....X..................X...............] duration: 11.57566509s FAIL (2025-10-09 15:46:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 11.575692202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.................X....X..................X..............] duration: 7.535986171s FAIL (2025-10-09 15:45:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 7.536008483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.................X....X..................X.............] duration: 7.136327275s FAIL (2025-10-09 15:44:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 7.136362092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X.................X....X..................X............] duration: 10.118098367s FAIL (2025-10-09 15:44:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.11814203s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X.................X....X..................X...........] duration: 6.997859373s FAIL (2025-10-09 15:43:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 6.997892766s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X.................X....X..................X..........] duration: 8.550460224s FAIL (2025-10-09 15:42:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 8.550494429s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X.................X....X..................X.........] duration: 8.797911705s FAIL (2025-10-09 15:42:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 8.797942813s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X.................X....X..................X........] duration: 14.757682044s FAIL (2025-10-09 15:41:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 14.757709648s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X.................X....X..................X.......] duration: 9.030902591s FAIL (2025-10-09 15:40:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 9.030932087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.................X....X..................X......] duration: 10.772937142s FAIL (2025-10-09 15:40:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.772971077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.................X....X..................X.....] duration: 19.772845101s FAIL (2025-10-09 15:40:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 19.772943508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.................X....X..................X....] duration: 10.105892075s FAIL (2025-10-09 15:39:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.105912112s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.................X....X..................X...] duration: 17.429917981s FAIL (2025-10-09 15:39:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 17.429968878s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.................X....X..................X..] duration: 21.893415531s FAIL (2025-10-09 15:38:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 21.893448995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X...............X.................X....X..................X.] duration: 10.821041182s FAIL (2025-10-09 15:37:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 10.821069627s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X...............X.................X....X..................X] duration: 30.002698428s FAIL (2025-10-09 15:37:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q) context deadline exceeded; piecestore: (Node ID: 12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, Piece ID: 3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ) context deadline exceeded ======LOGS====== Oct 9 15:37:02.242123523 (386h5m47.704215996s) [upload=339720]: Starting upload Oct 9 15:37:02.242222231 (386h5m47.704314714s) [upload=339720]: Done starting upload Oct 9 15:37:02.242341397 (386h5m47.70443388s) [upload=339720]: Uploading segments... Oct 9 15:37:02.242462188 (386h5m47.704554671s) [upload=339720]: Got next segment. Inline: false Oct 9 15:37:02.242528735 (386h5m47.704621218s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:37:02.242536409 (386h5m47.704628892s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:37:02.504197156 (386h5m47.966289629s) [seg_pos={0 0}, upload=339720]: Begin upload segment... Oct 9 15:37:02.504256378 (386h5m47.966348861s) [seg_pos={0 0}, upload=339720]: Done begin upload segment. Oct 9 15:37:02.504284772 (386h5m47.966377255s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506016159 (386h5m47.968108642s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.506805693 (386h5m47.968898176s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.507381258 (386h5m47.969473751s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.508983771 (386h5m47.971076254s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.509265788 (386h5m47.971358281s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510802103 (386h5m47.972894586s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.510991884 (386h5m47.973084377s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512377853 (386h5m47.974470336s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.512884819 (386h5m47.974977312s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.513772208 (386h5m47.975864691s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.514679907 (386h5m47.97677239s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.515172375 (386h5m47.977264858s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516618288 (386h5m47.978710771s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.516681799 (386h5m47.978774282s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:02.519789937 (386h5m47.98188243s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Uploading piece... Oct 9 15:37:04.342892789 (386h5m49.804985272s) [upload=339720]: Next returned nil segment Oct 9 15:37:04.342934217 (386h5m49.8050267s) [upload=339720]: Waiting for error group managing segments... Oct 9 15:37:04.771607028 (386h5m50.233699511s) [noise=true , address=45.91.116.10:4083 , piece_id=WWFIBCZ4S3SYN3NOUWPO276LIQJ7BM7X42G6Z2UY2O3EJDP5GGRA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.808771606 (386h5m50.270864089s) [noise=true , address=77.90.31.18:4807 , piece_id=3MVQ72K2DFLN2SZHDDGB5EK6WSKBIEOQAYKQKQQQLNQRIMZEEZOQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.911015099 (386h5m50.373107582s) [noise=true , address=77.90.31.19:4842 , piece_id=5GRKUSGWRVZZNJSQ7AVDAFRB2YRIOF3PQRCEMNENLLZMKAPIRT7Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:04.937504590 (386h5m50.399597073s) [noise=true , address=45.91.116.5:4299 , piece_id=EOURFB5N5DEUK6PRMWDZI7TCLXDK2DCY36DIR4ETE5FI67KCJ37A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.014747387 (386h5m50.47683987s) [noise=true , address=77.90.31.21:4878 , piece_id=NIOTO4IJZ6FRBGVDL6LYYI7OHWPPO4QEVWBATQ42AQZH5MPT5K4Q, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.181525045 (386h5m50.643617528s) [noise=true , address=77.90.31.17:4770 , piece_id=TCKYBCAFTDTNTLJCQRVEFCCHKT3DLNYILNR7MVOWFI4YHYBKPR6A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.385017486 (386h5m50.847109969s) [noise=true , address=145.79.227.7:4295 , piece_id=DXIJ26ILOHWTMS75FPHW7RRVB4CHHTWPRNQ5TQEE42GGH2TJVS2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.404928321 (386h5m50.867020804s) [noise=true , address=145.79.227.4:4185 , piece_id=UTDL5IOQ6GMNGPDMB3NQ4HCAYVODWE3JIOXBOK5H7OSKLON4FCQQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.456562682 (386h5m50.918655175s) [noise=true , address=145.79.227.11:4730 , piece_id=HBZCZVD5H27SWWTMVDWWNHOJZCVQC2YI6DX4DY2A3GDOQ54PMD2A, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.515032722 (386h5m50.977125205s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZK3543VGODZ3452ZZ23YLH5FARCRKGCOVVHAQSLBDTYCINKBJVQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:05.926564111 (386h5m51.388656595s) [noise=true , address=145.79.227.12:4767 , piece_id=EHEIO5WX5PQKX5ZDHPZ2XMAZTCTZAAII4L2Q4DJAI3GG2DX3XUXQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.373844666 (386h5m51.835937149s) [noise=true , address=145.79.227.13:4804 , piece_id=FUN5LPTLV2JQNLJB4JGNFWZIAVQTGEBJZULLVEP73PSHFSM3NZAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736384436 (386h5m52.198476909s) [noise=true , address=145.79.227.17:4970 , piece_id=2F5TXHEN3LIB2XN4DTSQN6W75BSTZYK3EL5JDKOMWI7VL27PTARA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: <nil> Oct 9 15:37:06.736427608 (386h5m52.198520091s) [seg_pos={0 0}, upload=339720]: Segment reached optimal threshold of 13 pieces. Oct 9 15:37:06.736566292 (386h5m52.198658775s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736576751 (386h5m52.198669234s) [noise=true , address=145.79.227.3:4151 , piece_id=MS3KJ3OCWQR5N6JWZCXIFTB4TFPLZNTNSQSWOJQP476OAKM6O7NQ, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736597040 (386h5m52.198689543s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736607961 (386h5m52.198700444s) [noise=true , address=145.79.227.6:4259 , piece_id=NGTT3BQRHDUOQ4QTTSK6GE2CI4J7AXZ5LXKBI245O3PT35KRP7NA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736799205 (386h5m52.198891699s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Oct 9 15:37:06.736808032 (386h5m52.198900505s) [noise=true , address=145.79.227.15:4876 , piece_id=KLXHO4LI22RXA7EZFM5HWPLOPPTWFGJXBJJ4MLV4ZTSMBKZ5DPAA, seg_pos={0 0}, upload=339720]: Long tail cancellation due to optimal threshold Oct 9 15:37:06.736843279 (386h5m52.198935762s) [seg_pos={0 0}, upload=339720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:37:06.736970692 (386h5m52.199063185s) [upload=339720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:37:06.736978837 (386h5m52.19907132s) [upload=339720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:37:07.042768755 (386h5m52.504861238s) [upload=339720]: Done uploading segments... Oct 9 15:37:07.042792771 (386h5m52.504885254s) [upload=339720]: Upload finished. err: <nil> Duration: 30.00273034s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X...............X.................X....X..................] duration: 15.521327165s FAIL (2025-10-09 15:36:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 15.52136086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X...............X.................X....X.................] duration: 18.683113988s FAIL (2025-10-09 15:35:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 18.683148042s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X...............X.................X....X................] duration: 19.982774491s FAIL (2025-10-09 15:35:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 19.982794778s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X...............X.................X....X...............] duration: 27.904716414s FAIL (2025-10-09 15:34:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 27.904743475s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.....X...............X.................X....X..............] duration: 27.40430592s FAIL (2025-10-09 15:34:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 27.404340806s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.....X...............X.................X....X.............] duration: 8.662691232s FAIL (2025-10-09 15:33:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 8.66283246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.....X...............X.................X....X............] duration: 9.216154135s FAIL (2025-10-09 15:33:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 9.216203319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..X.....X...............X.................X....X...........] duration: 10.758285442s FAIL (2025-10-09 15:32:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 10.758320187s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X..X.....X...............X.................X....X..........] duration: 8.746872364s FAIL (2025-10-09 15:31:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 8.746961646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X..X.....X...............X.................X....X.........] duration: 25.918232401s FAIL (2025-10-09 15:31:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 25.918267299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X..X.....X...............X.................X....X........] duration: 25.154985578s FAIL (2025-10-09 15:30:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q) context deadline exceeded ======LOGS====== Oct 9 15:24:06.709402947 (385h52m52.17149543s) [upload=339531]: Starting upload Oct 9 15:24:06.709492177 (385h52m52.17158466s) [upload=339531]: Done starting upload Oct 9 15:24:06.709584853 (385h52m52.171677346s) [upload=339531]: Uploading segments... Oct 9 15:24:06.709788992 (385h52m52.171881475s) [upload=339531]: Got next segment. Inline: false Oct 9 15:24:06.709948025 (385h52m52.172040508s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 15:24:06.709955359 (385h52m52.172047842s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 15:24:06.961133614 (385h52m52.423226097s) [seg_pos={0 0}, upload=339531]: Begin upload segment... Oct 9 15:24:06.961215950 (385h52m52.423308434s) [seg_pos={0 0}, upload=339531]: Done begin upload segment. Oct 9 15:24:06.961248853 (385h52m52.423341336s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961260055 (385h52m52.423352538s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.961364324 (385h52m52.423456807s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962261922 (385h52m52.424354405s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.962996561 (385h52m52.425089054s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963450747 (385h52m52.42554323s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.963946050 (385h52m52.426038533s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.964519150 (385h52m52.426611633s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.965515518 (385h52m52.427608001s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966306074 (385h52m52.428398557s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.966973104 (385h52m52.429065587s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.968336800 (385h52m52.430429283s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.970435927 (385h52m52.43252841s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.973260746 (385h52m52.435353229s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.975012342 (385h52m52.437104825s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:06.976159236 (385h52m52.438251719s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Uploading piece... Oct 9 15:24:08.939867009 (385h52m54.401959482s) [upload=339531]: Next returned nil segment Oct 9 15:24:08.939888049 (385h52m54.401980532s) [upload=339531]: Waiting for error group managing segments... Oct 9 15:24:09.278935654 (385h52m54.741028127s) [noise=true , address=45.91.116.11:4117 , piece_id=5PVQKIV3R6XV764VD7FZDQHYBYKG67GUEOAH6J37ZHOL3HPXZPSA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630153916 (385h52m55.092246399s) [noise=true , address=45.91.116.7:4009 , piece_id=POXFHWYC3IK63TQOR3KHGMQMMMEJNOALYX2634J5DWLDYSRA3M3Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.630389264 (385h52m55.092481747s) [noise=true , address=77.90.31.10:4518 , piece_id=6OUXLJ4SXLMU5JJ442TE2JUSF6MJGST2LDVIGIWCMZS4WBU6KYFA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.637709484 (385h52m55.099801967s) [noise=true , address=77.90.31.14:4661 , piece_id=WSFLOIV2W57JQHQNOQR3WMFFSGORNWRFQTREMK7WES3NAVPCFFEQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.687974609 (385h52m55.150067092s) [noise=true , address=145.79.227.2:4114 , piece_id=4A2MH77YWFCB2AHQZC2RBWZCDWM2G7JG6FATBTTSMMLZZBOBNKCQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.745324645 (385h52m55.207417128s) [noise=true , address=45.91.116.4:4939 , piece_id=MMFYWSAGV5ENKWLKBAUI3PYPDBK47EHL5S5T4FKKXB2IKMGFFE4A, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.764355744 (385h52m55.226448227s) [noise=true , address=145.79.227.6:4257 , piece_id=VQFBVD6CFAQEJYJWZ64M4CBOL62ZLXOHZMOYSGTBKZZSWXR23UTA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.774708318 (385h52m55.236800801s) [noise=true , address=145.79.227.16:4911 , piece_id=POANCP5TEYALGG35CYEAOPCFVVXA26N4FBR7KYEPQTOWSVHVH5EQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.784869036 (385h52m55.246961519s) [noise=true , address=77.90.31.17:4770 , piece_id=IXCQGIAE3DK6ARUYYJCOQMRH744WNURNSQ62CDNOESDFBGJSECLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.819998149 (385h52m55.282090632s) [noise=true , address=145.79.227.5:4223 , piece_id=JQL2TC5P3K7Q62RF63DKZ5A5GN22X6GA7ZYK3YUOLQFYLKRJNSWQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.848346901 (385h52m55.310439384s) [noise=true , address=145.79.227.3:4151 , piece_id=QE3W2FEZGXDZDBPKJZCPY4P5SL4POMJHALWPQ4CNV4KZ5BVAIBLA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926459603 (385h52m55.388552086s) [noise=true , address=145.79.227.4:4185 , piece_id=25PMOVW3EFQQBTKI75VMYTSBTAFXC2YXYNSVVT7BRL6QHJCMHZ6Q, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926710391 (385h52m55.388802874s) [noise=true , address=145.79.227.11:4732 , piece_id=DVIYPZHWAM7ZKP62IPVD675MUMKTGKSDPLTDFUILYM44L4IGJBBA, seg_pos={0 0}, upload=339531]: Done uploading piece. err: <nil> Oct 9 15:24:09.926734697 (385h52m55.38882718s) [seg_pos={0 0}, upload=339531]: Segment reached optimal threshold of 13 pieces. Oct 9 15:24:09.926864134 (385h52m55.388956617s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926870997 (385h52m55.38896348s) [noise=true , address=77.90.31.16:4733 , piece_id=VY572F4WFW2IGZONGBZ6PAKLNLJZL6Q5GZZK4B5YQ27RZT2O27VQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926895734 (385h52m55.388988207s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926900643 (385h52m55.388993126s) [noise=true , address=145.79.227.16:4910 , piece_id=ULXSTGNUDRQ5DCUMYJDIQQS7OKIKQ6NRQGJRJ3A4DFEW63GEJ5SQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926926984 (385h52m55.389019467s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 9 15:24:09.926932314 (385h52m55.389024797s) [noise=true , address=145.79.227.15:4875 , piece_id=PZPKZIZ7VHTIDE4MBOV25ZBO6BJNCMPXRZKPLBFZGWMUK4K7FSXQ, seg_pos={0 0}, upload=339531]: Long tail cancellation due to optimal threshold Oct 9 15:24:09.926953844 (385h52m55.389046327s) [seg_pos={0 0}, upload=339531]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 15:24:09.927093059 (385h52m55.389185542s) [upload=339531]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 15:24:09.927102508 (385h52m55.389194991s) [upload=339531]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 15:24:10.255867965 (385h52m55.717960448s) [upload=339531]: Done uploading segments... Oct 9 15:24:10.255889877 (385h52m55.71798236s) [upload=339531]: Upload finished. err: <nil> Duration: 25.155016898s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)