Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.......X.X........] duration: 5.834586534s FAIL (2025-10-26 13:53:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 5.83462136s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................X....X.....................] duration: 5.741794745s FAIL (2025-10-26 13:53:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.741822109s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.......X.X.......] duration: 6.038123326s FAIL (2025-10-26 13:52:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 6.038151249s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................X....X....................] duration: 5.723662211s FAIL (2025-10-26 13:52:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.723691986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.......X.X......] duration: 6.028607204s FAIL (2025-10-26 13:51:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 6.028639036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.......X.X.....] duration: 7.820347631s FAIL (2025-10-26 13:51:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 7.820379982s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................X....X...................] duration: 5.708613622s FAIL (2025-10-26 13:51:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.70868095s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X.......X.X....] duration: 6.098362404s FAIL (2025-10-26 13:51:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 6.098399405s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................X....X..................] duration: 11.856540342s FAIL (2025-10-26 13:50:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.856568957s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X.......X.X...] duration: 6.924463203s FAIL (2025-10-26 13:50:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 6.924493589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X.......X.X..] duration: 7.335454287s FAIL (2025-10-26 13:49:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 7.335483564s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X....X.................] duration: 5.615981769s FAIL (2025-10-26 13:49:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.616007767s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................X.......X.X.] duration: 6.772139783s FAIL (2025-10-26 13:48:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 6.772174549s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................X....X................] duration: 5.691523133s FAIL (2025-10-26 13:48:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.691552648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X.......X.X] duration: 30.003223595s FAIL (2025-10-26 13:48:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ) context deadline exceeded; piecestore: (Node ID: 12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, Piece ID: C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA) context deadline exceeded ======LOGS====== 46.163905297 (792h16m31.62599778s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:47:46.415507525 (792h16m31.877600008s) [seg_pos={0 0}, upload=696923]: Begin upload segment... Oct 26 13:47:46.415570155 (792h16m31.877662638s) [seg_pos={0 0}, upload=696923]: Done begin upload segment. Oct 26 13:47:46.415601114 (792h16m31.877693587s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.415609439 (792h16m31.877701922s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416219802 (792h16m31.878312275s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416325654 (792h16m31.878418127s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416790368 (792h16m31.878882841s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.416898424 (792h16m31.878990907s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417153720 (792h16m31.879246203s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417479861 (792h16m31.879572334s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.417580132 (792h16m31.879672615s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418046931 (792h16m31.880139404s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418075976 (792h16m31.880168459s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418561250 (792h16m31.880653723s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.418583743 (792h16m31.880676226s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419167324 (792h16m31.881259807s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.419776234 (792h16m31.881868717s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:46.420331061 (792h16m31.882423544s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Uploading piece... Oct 26 13:47:48.164291102 (792h16m33.626383585s) [upload=696923]: Next returned nil segment Oct 26 13:47:48.164311901 (792h16m33.626404384s) [upload=696923]: Waiting for error group managing segments... Oct 26 13:47:48.445892354 (792h16m33.907984827s) [noise=true , address=145.79.227.13:4801 , piece_id=SHHQPXNSLTCIQTCN2YKQXIPXHXFJWTLEQQWXFRRVUO4LKZUUII6Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.450335054 (792h16m33.912427537s) [noise=true , address=145.79.227.15:4876 , piece_id=S35OYD4JPNT3B5HOBK52HIWVRYU3YDQFL6NFIJAPCX2CSUMNNUFQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.453409137 (792h16m33.91550161s) [noise=true , address=145.79.227.23:4545 , piece_id=LQBBNGUF4NPZSKKRFLHX5SKJXZOXVDJP5HK5G7PS4DF2ULHSU2RQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.460362688 (792h16m33.922455171s) [noise=true , address=145.79.227.22:4511 , piece_id=CQTBNFS3II3CH6WKVZF6DCNFEY24MNA25XPVCEJODHYVGJPMNBKQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.465456998 (792h16m33.927549472s) [noise=true , address=145.79.227.17:4971 , piece_id=N36JVGZXZXW5ZLW34U7OQVW5EOGEU7ZMUB6AZUSXLLP6OWPNUX7A, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.483352365 (792h16m33.945444838s) [noise=true , address=145.79.227.24:4583 , piece_id=T6PBGXVYE6W4FCSF7BFRAPRARZVDYIPOA2NWA7ICWB5IOEQJBVFA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.485927960 (792h16m33.948020443s) [noise=true , address=145.79.227.16:4911 , piece_id=IEMZXRFII4YRQXO3OPE7TFL2DSY56ILH3QAMDZASUQJNXSVD5CVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.495117638 (792h16m33.957210121s) [noise=true , address=145.79.227.8:4329 , piece_id=GXVGLDCHCRDPJU3PZMAY2L7FAYOA2RSKEFXRIRAH4XZI6Z536KCQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.519274919 (792h16m33.981367392s) [noise=true , address=145.79.227.9:4367 , piece_id=ADMPXE7P3DD5YUMPQ45U232EPMH45FTPLJPXBSFACTRCQKCQ6EVQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.707682332 (792h16m34.169774815s) [noise=true , address=77.90.31.18:4807 , piece_id=C2QLRDTBWJPZAAJFW532LXSMNM7TK2AVGLK3X7NYILCCYD2HIFJA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.716174933 (792h16m34.178267416s) [noise=true , address=77.90.31.19:4843 , piece_id=VKRUCH25NDBQXIPU6ELBC5MWRNZHLVPMS7GUUY42AGL3VQEMPIIA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.751215507 (792h16m34.21330798s) [noise=true , address=77.90.31.13:4627 , piece_id=HZPRWSTMN6G4RKMQM2HSZWS324DE5B53LK64N3ARWWZNYAVOEEQQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800219771 (792h16m34.262312244s) [noise=true , address=77.90.31.10:4519 , piece_id=UIO56OMGKB6UIOPJH7NYGT6CMTPGJVWWTYQJF6QNN3X4ETSANZRQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: <nil> Oct 26 13:47:48.800251962 (792h16m34.262344435s) [seg_pos={0 0}, upload=696923]: Segment reached optimal threshold of 13 pieces. Oct 26 13:47:48.800440811 (792h16m34.262533285s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35544->77.90.31.21:4878: use of closed network connection Oct 26 13:47:48.800451963 (792h16m34.262544436s) [noise=true , address=77.90.31.21:4878 , piece_id=XIDVOPLYOMVZRXIWAQTRFXIRX2XVBDOMGSGTY6M6TR6UBOPFC6PQ, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800509893 (792h16m34.262602376s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43428->77.90.31.11:4553: use of closed network connection Oct 26 13:47:48.800521125 (792h16m34.262613598s) [noise=true , address=77.90.31.11:4553 , piece_id=OSKXDLRMJGG536GZZWXYDTJOAXUV6C4PXVJDNAEB5IY6JQCPXR2Q, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800560450 (792h16m34.262652933s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41842->77.90.31.17:4772: use of closed network connection Oct 26 13:47:48.800568795 (792h16m34.262661279s) [noise=true , address=77.90.31.17:4772 , piece_id=A46PDJCIELRNQWHQDLAKXBYLMOXQC4SM6WHJ7ANXUO3A6PHWRFOA, seg_pos={0 0}, upload=696923]: Long tail cancellation due to optimal threshold Oct 26 13:47:48.800590437 (792h16m34.26268292s) [seg_pos={0 0}, upload=696923]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:47:48.800731355 (792h16m34.262823838s) [upload=696923]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:47:48.800739050 (792h16m34.262831523s) [upload=696923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:47:49.092388375 (792h16m34.554480858s) [upload=696923]: Done uploading segments... Oct 26 13:47:49.092408874 (792h16m34.554501347s) [upload=696923]: Upload finished. err: <nil> Duration:* 30.003248833s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X....X...............] duration: 5.601605379s FAIL (2025-10-26 13:48:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.601633261s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.......X.] duration: 6.147795871s FAIL (2025-10-26 13:47:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q) context deadline exceeded; piecestore: (Node ID: 1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, Piece ID: SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A) context deadline exceeded ======LOGS====== Oct 26 13:46:14.275830024 (792h14m59.737922507s) [upload=696898]: Starting upload Oct 26 13:46:14.275923612 (792h14m59.738016095s) [upload=696898]: Done starting upload Oct 26 13:46:14.276004777 (792h14m59.73809725s) [upload=696898]: Uploading segments... Oct 26 13:46:14.276273208 (792h14m59.738365681s) [upload=696898]: Got next segment. Inline: false Oct 26 13:46:14.276336268 (792h14m59.738428751s) [upload=696898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:46:14.276345295 (792h14m59.738437768s) [upload=696898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:46:14.519985654 (792h14m59.982078137s) [seg_pos={0 0}, upload=696898]: Begin upload segment... Oct 26 13:46:14.520062069 (792h14m59.982154552s) [seg_pos={0 0}, upload=696898]: Done begin upload segment. Oct 26 13:46:14.520090984 (792h14m59.982183467s) [noise=true , address=145.79.227.13:4804 , piece_id=UZPULBD5P5ANVECKC2NFY3RSB2FEJSX6J7FNTEGJHR2VU3D3AVIQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.520102145 (792h14m59.982194628s) [noise=true , address=145.79.227.16:4911 , piece_id=JJ2FAR2OR2DPAR547WLPO3F5TAZWPEPSVEF72227CP26RROQ4E4A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.520179453 (792h14m59.982271936s) [noise=true , address=77.90.31.21:4879 , piece_id=4CXZQ7AEETXRJ5MNVNH5R7V7CPUJVX6M64LPJCK6ZF7W6M2MHHXA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.521087372 (792h14m59.983179855s) [noise=true , address=145.79.227.5:4223 , piece_id=6TIZG4IT62CBGZMYIMQH5YCCLK65GF44ZVPXQLR5ZGIOVSPPC7IA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.521684098 (792h14m59.983776571s) [noise=true , address=77.90.31.13:4628 , piece_id=SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.522140527 (792h14m59.98423301s) [noise=true , address=45.91.116.7:4009 , piece_id=UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.522553173 (792h14m59.984645656s) [noise=true , address=145.79.227.5:4222 , piece_id=TVTELPBBKE5A7AMRD5VSHDO6QWT4UUX6DL6XXY2XDZHGVA4MG25A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.523156722 (792h14m59.985249205s) [noise=true , address=145.79.227.26:4654 , piece_id=FSD2ZE7EFBKWMF56IYWKRZLIGBI4GQH5OO3HQZEOMAF6TP53M6QA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.523703754 (792h14m59.985796237s) [noise=true , address=145.79.227.18:4006 , piece_id=EKAOIQMUDN2GXU62ACPUD4TG5Q3VEVK4UIXHZUP4GDALVLFW5AXA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.524178467 (792h14m59.98627095s) [noise=true , address=145.79.227.24:4581 , piece_id=IAIOFYHT244UMU472E72HSU4YUECCPKM54ET6RCWBOB2ODVAN7MQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.525944450 (792h14m59.988036933s) [noise=true , address=77.90.31.19:4843 , piece_id=2LSU2EHFTWKS3BIGFGUN2C56SEAAX477HX766WT5CDQKHEU3JRVQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.526560864 (792h14m59.988653347s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.527147291 (792h14m59.989239774s) [noise=true , address=145.79.227.23:4546 , piece_id=SL6SRFBKGNECLNEG5O4LL7Z3G3XVBUZALL7JBXDJ3TLRFGSFQLSQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.527649046 (792h14m59.989741519s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.528262374 (792h14m59.990354857s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.529029124 (792h14m59.991121597s) [noise=true , address=145.79.227.21:4473 , piece_id=N6ERXXUAUKXZHJ6X3KYWPLSFHVY3Q6L6Z4HZ36E64DXYZAXO3FJA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:16.150290655 (792h15m1.612383138s) [upload=696898]: Next returned nil segment Oct 26 13:46:16.150315853 (792h15m1.612408346s) [upload=696898]: Waiting for error group managing segments... Oct 26 13:46:16.537690918 (792h15m1.999783402s) [noise=true , address=145.79.227.13:4804 , piece_id=UZPULBD5P5ANVECKC2NFY3RSB2FEJSX6J7FNTEGJHR2VU3D3AVIQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.538171033 (792h15m2.000263516s) [noise=true , address=145.79.227.18:4006 , piece_id=EKAOIQMUDN2GXU62ACPUD4TG5Q3VEVK4UIXHZUP4GDALVLFW5AXA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.558621435 (792h15m2.020713918s) [noise=true , address=145.79.227.5:4222 , piece_id=TVTELPBBKE5A7AMRD5VSHDO6QWT4UUX6DL6XXY2XDZHGVA4MG25A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.563288131 (792h15m2.025380614s) [noise=true , address=145.79.227.24:4581 , piece_id=IAIOFYHT244UMU472E72HSU4YUECCPKM54ET6RCWBOB2ODVAN7MQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.564535416 (792h15m2.026627899s) [noise=true , address=145.79.227.21:4473 , piece_id=N6ERXXUAUKXZHJ6X3KYWPLSFHVY3Q6L6Z4HZ36E64DXYZAXO3FJA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.566532138 (792h15m2.028624611s) [noise=true , address=145.79.227.16:4911 , piece_id=JJ2FAR2OR2DPAR547WLPO3F5TAZWPEPSVEF72227CP26RROQ4E4A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.571447378 (792h15m2.033539861s) [noise=true , address=145.79.227.23:4546 , piece_id=SL6SRFBKGNECLNEG5O4LL7Z3G3XVBUZALL7JBXDJ3TLRFGSFQLSQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.603606755 (792h15m2.065699228s) [noise=true , address=145.79.227.5:4223 , piece_id=6TIZG4IT62CBGZMYIMQH5YCCLK65GF44ZVPXQLR5ZGIOVSPPC7IA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.615406354 (792h15m2.077498827s) [noise=true , address=145.79.227.26:4654 , piece_id=FSD2ZE7EFBKWMF56IYWKRZLIGBI4GQH5OO3HQZEOMAF6TP53M6QA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.745588857 (792h15m2.20768135s) [noise=true , address=77.90.31.21:4879 , piece_id=4CXZQ7AEETXRJ5MNVNH5R7V7CPUJVX6M64LPJCK6ZF7W6M2MHHXA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.773527968 (792h15m2.235620441s) [noise=true , address=77.90.31.19:4843 , piece_id=2LSU2EHFTWKS3BIGFGUN2C56SEAAX477HX766WT5CDQKHEU3JRVQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.847166219 (792h15m2.309258692s) [noise=true , address=45.91.116.7:4009 , piece_id=UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.875426653 (792h15m2.337519136s) [noise=true , address=77.90.31.13:4628 , piece_id=SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.875468061 (792h15m2.337560544s) [seg_pos={0 0}, upload=696898]: Segment reached optimal threshold of 13 pieces. Oct 26 13:46:16.875664334 (792h15m2.337756807s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 26 13:46:16.875679955 (792h15m2.337772428s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875693580 (792h15m2.337786053s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection Oct 26 13:46:16.875701685 (792h15m2.337794168s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875767741 (792h15m2.337860224s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Oct 26 13:46:16.875777901 (792h15m2.337870384s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875798811 (792h15m2.337891294s) [seg_pos={0 0}, upload=696898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:46:16.875886288 (792h15m2.337978761s) [upload=696898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:46:16.875962242 (792h15m2.338054715s) [upload=696898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:46:17.168883882 (792h15m2.630976355s) [upload=696898]: Done uploading segments... Oct 26 13:46:17.168907206 (792h15m2.630999679s) [upload=696898]: Upload finished. err: <nil> Duration: 6.147825939s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................X....X..............] duration: 5.699062209s FAIL (2025-10-26 13:47:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.699102314s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X....X.............] duration: 5.734123121s FAIL (2025-10-26 13:46:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.734148268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.......X] duration: 30.002998919s FAIL (2025-10-26 13:46:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q) context deadline exceeded; piecestore: (Node ID: 1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, Piece ID: SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A) context deadline exceeded ======LOGS====== Oct 26 13:46:14.275830024 (792h14m59.737922507s) [upload=696898]: Starting upload Oct 26 13:46:14.275923612 (792h14m59.738016095s) [upload=696898]: Done starting upload Oct 26 13:46:14.276004777 (792h14m59.73809725s) [upload=696898]: Uploading segments... Oct 26 13:46:14.276273208 (792h14m59.738365681s) [upload=696898]: Got next segment. Inline: false Oct 26 13:46:14.276336268 (792h14m59.738428751s) [upload=696898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:46:14.276345295 (792h14m59.738437768s) [upload=696898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:46:14.519985654 (792h14m59.982078137s) [seg_pos={0 0}, upload=696898]: Begin upload segment... Oct 26 13:46:14.520062069 (792h14m59.982154552s) [seg_pos={0 0}, upload=696898]: Done begin upload segment. Oct 26 13:46:14.520090984 (792h14m59.982183467s) [noise=true , address=145.79.227.13:4804 , piece_id=UZPULBD5P5ANVECKC2NFY3RSB2FEJSX6J7FNTEGJHR2VU3D3AVIQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.520102145 (792h14m59.982194628s) [noise=true , address=145.79.227.16:4911 , piece_id=JJ2FAR2OR2DPAR547WLPO3F5TAZWPEPSVEF72227CP26RROQ4E4A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.520179453 (792h14m59.982271936s) [noise=true , address=77.90.31.21:4879 , piece_id=4CXZQ7AEETXRJ5MNVNH5R7V7CPUJVX6M64LPJCK6ZF7W6M2MHHXA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.521087372 (792h14m59.983179855s) [noise=true , address=145.79.227.5:4223 , piece_id=6TIZG4IT62CBGZMYIMQH5YCCLK65GF44ZVPXQLR5ZGIOVSPPC7IA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.521684098 (792h14m59.983776571s) [noise=true , address=77.90.31.13:4628 , piece_id=SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.522140527 (792h14m59.98423301s) [noise=true , address=45.91.116.7:4009 , piece_id=UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.522553173 (792h14m59.984645656s) [noise=true , address=145.79.227.5:4222 , piece_id=TVTELPBBKE5A7AMRD5VSHDO6QWT4UUX6DL6XXY2XDZHGVA4MG25A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.523156722 (792h14m59.985249205s) [noise=true , address=145.79.227.26:4654 , piece_id=FSD2ZE7EFBKWMF56IYWKRZLIGBI4GQH5OO3HQZEOMAF6TP53M6QA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.523703754 (792h14m59.985796237s) [noise=true , address=145.79.227.18:4006 , piece_id=EKAOIQMUDN2GXU62ACPUD4TG5Q3VEVK4UIXHZUP4GDALVLFW5AXA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.524178467 (792h14m59.98627095s) [noise=true , address=145.79.227.24:4581 , piece_id=IAIOFYHT244UMU472E72HSU4YUECCPKM54ET6RCWBOB2ODVAN7MQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.525944450 (792h14m59.988036933s) [noise=true , address=77.90.31.19:4843 , piece_id=2LSU2EHFTWKS3BIGFGUN2C56SEAAX477HX766WT5CDQKHEU3JRVQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.526560864 (792h14m59.988653347s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.527147291 (792h14m59.989239774s) [noise=true , address=145.79.227.23:4546 , piece_id=SL6SRFBKGNECLNEG5O4LL7Z3G3XVBUZALL7JBXDJ3TLRFGSFQLSQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.527649046 (792h14m59.989741519s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.528262374 (792h14m59.990354857s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:14.529029124 (792h14m59.991121597s) [noise=true , address=145.79.227.21:4473 , piece_id=N6ERXXUAUKXZHJ6X3KYWPLSFHVY3Q6L6Z4HZ36E64DXYZAXO3FJA, seg_pos={0 0}, upload=696898]: Uploading piece... Oct 26 13:46:16.150290655 (792h15m1.612383138s) [upload=696898]: Next returned nil segment Oct 26 13:46:16.150315853 (792h15m1.612408346s) [upload=696898]: Waiting for error group managing segments... Oct 26 13:46:16.537690918 (792h15m1.999783402s) [noise=true , address=145.79.227.13:4804 , piece_id=UZPULBD5P5ANVECKC2NFY3RSB2FEJSX6J7FNTEGJHR2VU3D3AVIQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.538171033 (792h15m2.000263516s) [noise=true , address=145.79.227.18:4006 , piece_id=EKAOIQMUDN2GXU62ACPUD4TG5Q3VEVK4UIXHZUP4GDALVLFW5AXA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.558621435 (792h15m2.020713918s) [noise=true , address=145.79.227.5:4222 , piece_id=TVTELPBBKE5A7AMRD5VSHDO6QWT4UUX6DL6XXY2XDZHGVA4MG25A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.563288131 (792h15m2.025380614s) [noise=true , address=145.79.227.24:4581 , piece_id=IAIOFYHT244UMU472E72HSU4YUECCPKM54ET6RCWBOB2ODVAN7MQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.564535416 (792h15m2.026627899s) [noise=true , address=145.79.227.21:4473 , piece_id=N6ERXXUAUKXZHJ6X3KYWPLSFHVY3Q6L6Z4HZ36E64DXYZAXO3FJA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.566532138 (792h15m2.028624611s) [noise=true , address=145.79.227.16:4911 , piece_id=JJ2FAR2OR2DPAR547WLPO3F5TAZWPEPSVEF72227CP26RROQ4E4A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.571447378 (792h15m2.033539861s) [noise=true , address=145.79.227.23:4546 , piece_id=SL6SRFBKGNECLNEG5O4LL7Z3G3XVBUZALL7JBXDJ3TLRFGSFQLSQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.603606755 (792h15m2.065699228s) [noise=true , address=145.79.227.5:4223 , piece_id=6TIZG4IT62CBGZMYIMQH5YCCLK65GF44ZVPXQLR5ZGIOVSPPC7IA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.615406354 (792h15m2.077498827s) [noise=true , address=145.79.227.26:4654 , piece_id=FSD2ZE7EFBKWMF56IYWKRZLIGBI4GQH5OO3HQZEOMAF6TP53M6QA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.745588857 (792h15m2.20768135s) [noise=true , address=77.90.31.21:4879 , piece_id=4CXZQ7AEETXRJ5MNVNH5R7V7CPUJVX6M64LPJCK6ZF7W6M2MHHXA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.773527968 (792h15m2.235620441s) [noise=true , address=77.90.31.19:4843 , piece_id=2LSU2EHFTWKS3BIGFGUN2C56SEAAX477HX766WT5CDQKHEU3JRVQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.847166219 (792h15m2.309258692s) [noise=true , address=45.91.116.7:4009 , piece_id=UZY5W6IBS57PGDZGEKMT3IPKP7PSWBHWDRT2D22ZOYP4XPM57P5Q, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.875426653 (792h15m2.337519136s) [noise=true , address=77.90.31.13:4628 , piece_id=SQ4NX7XWJONYTJIUM5I6BBNP7ON6HES2WERTZJPYZXODO2UKO25A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: <nil> Oct 26 13:46:16.875468061 (792h15m2.337560544s) [seg_pos={0 0}, upload=696898]: Segment reached optimal threshold of 13 pieces. Oct 26 13:46:16.875664334 (792h15m2.337756807s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 26 13:46:16.875679955 (792h15m2.337772428s) [noise=true , address=77.90.31.10:4520 , piece_id=5WU7UKK33NRZRWLIABXRGLPHEUH4YJSPI4EY2ZEDRFTCJ7JRF3FQ, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875693580 (792h15m2.337786053s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47934->77.90.31.12:4592: use of closed network connection Oct 26 13:46:16.875701685 (792h15m2.337794168s) [noise=true , address=77.90.31.12:4592 , piece_id=PISFOEN7LZLLOO747JTY3PBBCRDAOY4GZPGPP7MIOTQQHC6GHX5A, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875767741 (792h15m2.337860224s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Oct 26 13:46:16.875777901 (792h15m2.337870384s) [noise=true , address=77.90.31.18:4805 , piece_id=5SCKNDTBWATF4NYBNJVCWICTVB5NASG5A3QYQM5BVLZMS5ZLLFGA, seg_pos={0 0}, upload=696898]: Long tail cancellation due to optimal threshold Oct 26 13:46:16.875798811 (792h15m2.337891294s) [seg_pos={0 0}, upload=696898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:46:16.875886288 (792h15m2.337978761s) [upload=696898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:46:16.875962242 (792h15m2.338054715s) [upload=696898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:46:17.168883882 (792h15m2.630976355s) [upload=696898]: Done uploading segments... Oct 26 13:46:17.168907206 (792h15m2.630999679s) [upload=696898]: Upload finished. err: <nil> Duration: 30.00303069s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X....X............] duration: 5.681016794s FAIL (2025-10-26 13:45:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.681044337s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.053146149s FAIL (2025-10-26 13:45:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 6.053176548s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X....X...........] duration: 5.765747541s FAIL (2025-10-26 13:44:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.765794389s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.115035544s FAIL (2025-10-26 13:44:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 6.115069939s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................X....X..........] duration: 5.69468919s FAIL (2025-10-26 13:43:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.694732312s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.88264882s FAIL (2025-10-26 13:43:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 5.882697313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.016301271s FAIL (2025-10-26 13:42:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 6.016331438s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X....X.........] duration: 5.711249753s FAIL (2025-10-26 13:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.711272747s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X....X........] duration: 5.712624143s FAIL (2025-10-26 13:42:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.712735836s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.844214667s FAIL (2025-10-26 13:41:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 5.844253471s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X....X.......] duration: 5.668956295s FAIL (2025-10-26 13:41:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.668983386s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................X....X......] duration: 5.641767107s FAIL (2025-10-26 13:40:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.641797285s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.00244113s FAIL (2025-10-26 13:40:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 6.002475145s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration: 15.07742163s FAIL (2025-10-26 13:40:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.077437499s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X....X....] duration: 10.320759921s FAIL (2025-10-26 13:39:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.320781081s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.045157957s FAIL (2025-10-26 13:39:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 6.045222751s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002347757s FAIL (2025-10-26 13:39:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 26 13:38:53.807478506 (792h7m39.269570989s) [upload=696789]: Starting upload Oct 26 13:38:53.807570702 (792h7m39.269663185s) [upload=696789]: Done starting upload Oct 26 13:38:53.807673628 (792h7m39.269766101s) [upload=696789]: Uploading segments... Oct 26 13:38:53.807801982 (792h7m39.269894455s) [upload=696789]: Got next segment. Inline: false Oct 26 13:38:53.807844303 (792h7m39.269936786s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 13:38:53.807850985 (792h7m39.269943468s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 13:38:54.070601132 (792h7m39.532693615s) [seg_pos={0 0}, upload=696789]: Begin upload segment... Oct 26 13:38:54.070681034 (792h7m39.532773507s) [seg_pos={0 0}, upload=696789]: Done begin upload segment. Oct 26 13:38:54.070717654 (792h7m39.532810128s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.070766858 (792h7m39.532859331s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071406996 (792h7m39.533499479s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.071873886 (792h7m39.533966369s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072347958 (792h7m39.534440431s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072361114 (792h7m39.534453597s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.072862288 (792h7m39.534954771s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073046198 (792h7m39.535138671s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073262170 (792h7m39.535354643s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073526643 (792h7m39.535619126s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.073735871 (792h7m39.535828344s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074171301 (792h7m39.536263774s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.074182702 (792h7m39.536275185s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.075032922 (792h7m39.537125405s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.076219832 (792h7m39.538312305s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:54.078227724 (792h7m39.540320197s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Uploading piece... Oct 26 13:38:55.775872132 (792h7m41.237964605s) [upload=696789]: Next returned nil segment Oct 26 13:38:55.775891318 (792h7m41.237983791s) [upload=696789]: Waiting for error group managing segments... Oct 26 13:38:56.089628374 (792h7m41.551720857s) [noise=true , address=145.79.227.5:4223 , piece_id=J272UXO2NF42SD3L7PBVZFGHHO7LG7PL5I23SSB3EP464GHGB3VQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.099480564 (792h7m41.561573038s) [noise=true , address=145.79.227.3:4151 , piece_id=BCTF6LNFTOXNDL65UZRVYADW3WMMINJCILBKJGWWBMXCPY5OFGAA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.105693384 (792h7m41.567785857s) [noise=true , address=145.79.227.26:4655 , piece_id=RJWNIWPMZ5XKK3USMSA6BJ6LJOHPYIDVMWEM3UQD2Q3KRSXXFDYA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.108951398 (792h7m41.571043872s) [noise=true , address=145.79.227.6:4259 , piece_id=ZHB376XUINCDMYL23FPUH3EPJKOTMKUWR7QOYWX2BNQJERJG7MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.111588911 (792h7m41.573681384s) [noise=true , address=145.79.227.21:4474 , piece_id=PQ6B2S5B5HCKBXG5E2K5DR7XIJBPVI7TPJFPPF5CSS534X3CJUEA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.172249300 (792h7m41.634341783s) [noise=true , address=145.79.227.17:4971 , piece_id=B4ID43SA674FGGFKYORVAI5A2XD3VFJGYYJ5VRH66A7WQSHSY5HQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.173186826 (792h7m41.635279309s) [noise=true , address=145.79.227.21:4475 , piece_id=DNMF7NTBTDLD2BC7FD24BRTSGCJIC3JTFGVLPLD4P7ADPBOHIAOA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.174129971 (792h7m41.636222454s) [noise=true , address=145.79.227.24:4581 , piece_id=Z5TSIJOA2KSDAMUX4XKWZIRMPQGDSK56JFHAUY3MY3AMP6XC5MNA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.198153957 (792h7m41.66024643s) [noise=true , address=145.79.227.15:4876 , piece_id=6KA3XVNZWJPBAJSUP2I5OEQ7P4LXJPFMPSRXP5S4DQJT3PBO6HPQ, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.202352903 (792h7m41.664445386s) [noise=true , address=145.79.227.17:4970 , piece_id=T2HPDQXRUOFYGEF6CKTCJWYJMBK5TGTCCGCOXBZYJHCVOR4GJDBA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.319573122 (792h7m41.781665595s) [noise=true , address=77.90.31.11:4555 , piece_id=BYUVPQ2XNWMVD6GQTG4WO6NSAYTRJ2VLGKEYGETFY25VQHEHSUIA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.381414771 (792h7m41.843507254s) [noise=true , address=77.90.31.15:4698 , piece_id=V6HT6J6O2ID5QKTTOHMZF7YK43IZXRBHCLHAL4RUFIM4JY5WOH5A, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396257344 (792h7m41.858349827s) [noise=true , address=77.90.31.19:4841 , piece_id=CUBNGIQGVXGFKATIKRYVS5FXLJO5BQVXIPFUAMFCKYPV6J2AJYCA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: <nil> Oct 26 13:38:56.396288564 (792h7m41.858381037s) [seg_pos={0 0}, upload=696789]: Segment reached optimal threshold of 13 pieces. Oct 26 13:38:56.396412589 (792h7m41.858505062s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396421506 (792h7m41.858513979s) [noise=true , address=77.90.31.12:4590 , piece_id=EEBQCFT5QZQVZIA6X7CJ2NXTTPGZK7RXQFZ3CRYRFNXKSQSAILJA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396515856 (792h7m41.858608329s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52740->45.91.116.12:4227: use of closed network connection Oct 26 13:38:56.396522428 (792h7m41.858614901s) [noise=true , address=45.91.116.12:4227 , piece_id=DFE7SYRAG46VYFDGFSZFK6KPHTW3KMFZWGUOFDVJYM4X53OSEI7Q, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396527788 (792h7m41.858620261s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Oct 26 13:38:56.396619273 (792h7m41.858711746s) [noise=true , address=77.90.31.16:4734 , piece_id=MUR4BEGJBZXAOSAEVD7XKKSPNIUOGJCM3L5ZVZLP6IDSKLAT75LA, seg_pos={0 0}, upload=696789]: Long tail cancellation due to optimal threshold Oct 26 13:38:56.396648359 (792h7m41.858740832s) [seg_pos={0 0}, upload=696789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 13:38:56.396731146 (792h7m41.858823619s) [upload=696789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 13:38:56.396737588 (792h7m41.858830071s) [upload=696789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 13:38:56.699305054 (792h7m42.161397527s) [upload=696789]: Done uploading segments... Oct 26 13:38:56.699324401 (792h7m42.161416884s) [upload=696789]: Upload finished. err: <nil> Duration: 30.002368015s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X....X...] duration: 5.79636128s FAIL (2025-10-26 13:38:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.796389202s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X....X..] duration: 5.605797817s FAIL (2025-10-26 13:37:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.605829527s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 5.852300149s FAIL (2025-10-26 13:37:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.852315608s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 1m0.001348764s FAIL (2025-10-26 13:37:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001378629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.677767604s FAIL (2025-10-26 13:35:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.677820055s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.611807412s FAIL (2025-10-26 13:34:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.6118325s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.841881919s FAIL (2025-10-26 13:34:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.84192413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.731324523s FAIL (2025-10-26 13:33:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.731342387s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000864343s FAIL (2025-10-26 13:33:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000893559s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.14536231s FAIL (2025-10-26 13:31:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, Piece ID: LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA) context deadline exceeded; piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ) context deadline exceeded ======LOGS====== Oct 26 12:46:53.403942988 (791h15m38.866035461s) [upload=696025]: Starting upload Oct 26 12:46:53.404013823 (791h15m38.866106296s) [upload=696025]: Done starting upload Oct 26 12:46:53.404117090 (791h15m38.866209573s) [upload=696025]: Uploading segments... Oct 26 12:46:53.404279107 (791h15m38.86637158s) [upload=696025]: Got next segment. Inline: false Oct 26 12:46:53.404317180 (791h15m38.866409663s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:46:53.404324474 (791h15m38.866416988s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:46:53.658593722 (791h15m39.120686205s) [seg_pos={0 0}, upload=696025]: Begin upload segment... Oct 26 12:46:53.658662202 (791h15m39.120754695s) [seg_pos={0 0}, upload=696025]: Done begin upload segment. Oct 26 12:46:53.658724461 (791h15m39.120816944s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658733939 (791h15m39.120826422s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658801477 (791h15m39.12089396s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659210807 (791h15m39.12130329s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659648500 (791h15m39.121740983s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660300091 (791h15m39.122392574s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660724689 (791h15m39.122817172s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661186418 (791h15m39.123278891s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661676462 (791h15m39.123768945s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.663003078 (791h15m39.125095561s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665783723 (791h15m39.127876206s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665918920 (791h15m39.128011433s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667324086 (791h15m39.129416559s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667824869 (791h15m39.129917362s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670075966 (791h15m39.132168449s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670197808 (791h15m39.132290291s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:55.395305932 (791h15m40.857398425s) [upload=696025]: Next returned nil segment Oct 26 12:46:55.395334627 (791h15m40.85742711s) [upload=696025]: Waiting for error group managing segments... Oct 26 12:46:55.679159905 (791h15m41.141252388s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.697302241 (791h15m41.159394714s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.698975928 (791h15m41.161068411s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.724922165 (791h15m41.187014648s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.726562328 (791h15m41.188654811s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.762781125 (791h15m41.224873618s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.770992812 (791h15m41.233085295s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.780498642 (791h15m41.242591126s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.791784423 (791h15m41.253876896s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.811267893 (791h15m41.273360377s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.937959549 (791h15m41.400052022s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.012483687 (791h15m41.47457616s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034087284 (791h15m41.496179767s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034126839 (791h15m41.496219322s) [seg_pos={0 0}, upload=696025]: Segment reached optimal threshold of 13 pieces. Oct 26 12:46:56.034284791 (791h15m41.496377274s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034292976 (791h15m41.496385459s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034307042 (791h15m41.496399515s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034317693 (791h15m41.496410176s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034378919 (791h15m41.496471392s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034386043 (791h15m41.496478526s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034404238 (791h15m41.496496721s) [seg_pos={0 0}, upload=696025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:46:56.034520208 (791h15m41.496612691s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:46:56.034527112 (791h15m41.496619585s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:46:56.358026346 (791h15m41.820118829s) [upload=696025]: Done uploading segments... Oct 26 12:46:56.358080729 (791h15m41.820173202s) [upload=696025]: Upload finished. err: <nil> Duration: 6.145393278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.213601696s FAIL (2025-10-26 13:30:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, Piece ID: LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA) context deadline exceeded; piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ) context deadline exceeded ======LOGS====== Oct 26 12:46:53.403942988 (791h15m38.866035461s) [upload=696025]: Starting upload Oct 26 12:46:53.404013823 (791h15m38.866106296s) [upload=696025]: Done starting upload Oct 26 12:46:53.404117090 (791h15m38.866209573s) [upload=696025]: Uploading segments... Oct 26 12:46:53.404279107 (791h15m38.86637158s) [upload=696025]: Got next segment. Inline: false Oct 26 12:46:53.404317180 (791h15m38.866409663s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:46:53.404324474 (791h15m38.866416988s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:46:53.658593722 (791h15m39.120686205s) [seg_pos={0 0}, upload=696025]: Begin upload segment... Oct 26 12:46:53.658662202 (791h15m39.120754695s) [seg_pos={0 0}, upload=696025]: Done begin upload segment. Oct 26 12:46:53.658724461 (791h15m39.120816944s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658733939 (791h15m39.120826422s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658801477 (791h15m39.12089396s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659210807 (791h15m39.12130329s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659648500 (791h15m39.121740983s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660300091 (791h15m39.122392574s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660724689 (791h15m39.122817172s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661186418 (791h15m39.123278891s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661676462 (791h15m39.123768945s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.663003078 (791h15m39.125095561s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665783723 (791h15m39.127876206s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665918920 (791h15m39.128011433s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667324086 (791h15m39.129416559s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667824869 (791h15m39.129917362s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670075966 (791h15m39.132168449s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670197808 (791h15m39.132290291s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:55.395305932 (791h15m40.857398425s) [upload=696025]: Next returned nil segment Oct 26 12:46:55.395334627 (791h15m40.85742711s) [upload=696025]: Waiting for error group managing segments... Oct 26 12:46:55.679159905 (791h15m41.141252388s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.697302241 (791h15m41.159394714s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.698975928 (791h15m41.161068411s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.724922165 (791h15m41.187014648s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.726562328 (791h15m41.188654811s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.762781125 (791h15m41.224873618s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.770992812 (791h15m41.233085295s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.780498642 (791h15m41.242591126s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.791784423 (791h15m41.253876896s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.811267893 (791h15m41.273360377s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.937959549 (791h15m41.400052022s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.012483687 (791h15m41.47457616s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034087284 (791h15m41.496179767s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034126839 (791h15m41.496219322s) [seg_pos={0 0}, upload=696025]: Segment reached optimal threshold of 13 pieces. Oct 26 12:46:56.034284791 (791h15m41.496377274s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034292976 (791h15m41.496385459s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034307042 (791h15m41.496399515s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034317693 (791h15m41.496410176s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034378919 (791h15m41.496471392s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034386043 (791h15m41.496478526s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034404238 (791h15m41.496496721s) [seg_pos={0 0}, upload=696025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:46:56.034520208 (791h15m41.496612691s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:46:56.034527112 (791h15m41.496619585s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:46:56.358026346 (791h15m41.820118829s) [upload=696025]: Done uploading segments... Oct 26 12:46:56.358080729 (791h15m41.820173202s) [upload=696025]: Upload finished. err: <nil> Duration: 6.213645159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.592428954s FAIL (2025-10-26 13:29:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, Piece ID: LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA) context deadline exceeded; piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ) context deadline exceeded ======LOGS====== Oct 26 12:46:53.403942988 (791h15m38.866035461s) [upload=696025]: Starting upload Oct 26 12:46:53.404013823 (791h15m38.866106296s) [upload=696025]: Done starting upload Oct 26 12:46:53.404117090 (791h15m38.866209573s) [upload=696025]: Uploading segments... Oct 26 12:46:53.404279107 (791h15m38.86637158s) [upload=696025]: Got next segment. Inline: false Oct 26 12:46:53.404317180 (791h15m38.866409663s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:46:53.404324474 (791h15m38.866416988s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:46:53.658593722 (791h15m39.120686205s) [seg_pos={0 0}, upload=696025]: Begin upload segment... Oct 26 12:46:53.658662202 (791h15m39.120754695s) [seg_pos={0 0}, upload=696025]: Done begin upload segment. Oct 26 12:46:53.658724461 (791h15m39.120816944s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658733939 (791h15m39.120826422s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658801477 (791h15m39.12089396s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659210807 (791h15m39.12130329s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659648500 (791h15m39.121740983s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660300091 (791h15m39.122392574s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660724689 (791h15m39.122817172s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661186418 (791h15m39.123278891s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661676462 (791h15m39.123768945s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.663003078 (791h15m39.125095561s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665783723 (791h15m39.127876206s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665918920 (791h15m39.128011433s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667324086 (791h15m39.129416559s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667824869 (791h15m39.129917362s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670075966 (791h15m39.132168449s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670197808 (791h15m39.132290291s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:55.395305932 (791h15m40.857398425s) [upload=696025]: Next returned nil segment Oct 26 12:46:55.395334627 (791h15m40.85742711s) [upload=696025]: Waiting for error group managing segments... Oct 26 12:46:55.679159905 (791h15m41.141252388s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.697302241 (791h15m41.159394714s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.698975928 (791h15m41.161068411s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.724922165 (791h15m41.187014648s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.726562328 (791h15m41.188654811s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.762781125 (791h15m41.224873618s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.770992812 (791h15m41.233085295s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.780498642 (791h15m41.242591126s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.791784423 (791h15m41.253876896s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.811267893 (791h15m41.273360377s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.937959549 (791h15m41.400052022s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.012483687 (791h15m41.47457616s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034087284 (791h15m41.496179767s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034126839 (791h15m41.496219322s) [seg_pos={0 0}, upload=696025]: Segment reached optimal threshold of 13 pieces. Oct 26 12:46:56.034284791 (791h15m41.496377274s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034292976 (791h15m41.496385459s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034307042 (791h15m41.496399515s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034317693 (791h15m41.496410176s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034378919 (791h15m41.496471392s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034386043 (791h15m41.496478526s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034404238 (791h15m41.496496721s) [seg_pos={0 0}, upload=696025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:46:56.034520208 (791h15m41.496612691s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:46:56.034527112 (791h15m41.496619585s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:46:56.358026346 (791h15m41.820118829s) [upload=696025]: Done uploading segments... Oct 26 12:46:56.358080729 (791h15m41.820173202s) [upload=696025]: Upload finished. err: <nil> Duration: 6.592462287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.275446279s FAIL (2025-10-26 13:28:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, Piece ID: LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA) context deadline exceeded; piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ) context deadline exceeded ======LOGS====== Oct 26 12:46:53.403942988 (791h15m38.866035461s) [upload=696025]: Starting upload Oct 26 12:46:53.404013823 (791h15m38.866106296s) [upload=696025]: Done starting upload Oct 26 12:46:53.404117090 (791h15m38.866209573s) [upload=696025]: Uploading segments... Oct 26 12:46:53.404279107 (791h15m38.86637158s) [upload=696025]: Got next segment. Inline: false Oct 26 12:46:53.404317180 (791h15m38.866409663s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 26 12:46:53.404324474 (791h15m38.866416988s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 26 12:46:53.658593722 (791h15m39.120686205s) [seg_pos={0 0}, upload=696025]: Begin upload segment... Oct 26 12:46:53.658662202 (791h15m39.120754695s) [seg_pos={0 0}, upload=696025]: Done begin upload segment. Oct 26 12:46:53.658724461 (791h15m39.120816944s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658733939 (791h15m39.120826422s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.658801477 (791h15m39.12089396s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659210807 (791h15m39.12130329s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.659648500 (791h15m39.121740983s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660300091 (791h15m39.122392574s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.660724689 (791h15m39.122817172s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661186418 (791h15m39.123278891s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.661676462 (791h15m39.123768945s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.663003078 (791h15m39.125095561s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665783723 (791h15m39.127876206s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.665918920 (791h15m39.128011433s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667324086 (791h15m39.129416559s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.667824869 (791h15m39.129917362s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670075966 (791h15m39.132168449s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:53.670197808 (791h15m39.132290291s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Uploading piece... Oct 26 12:46:55.395305932 (791h15m40.857398425s) [upload=696025]: Next returned nil segment Oct 26 12:46:55.395334627 (791h15m40.85742711s) [upload=696025]: Waiting for error group managing segments... Oct 26 12:46:55.679159905 (791h15m41.141252388s) [noise=true , address=145.79.227.3:4150 , piece_id=IXDGBEO7B5O72UXGSITT2ZTYIGWMMDJID4AQQHWJVDOITJXHWBLQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.697302241 (791h15m41.159394714s) [noise=true , address=145.79.227.9:4366 , piece_id=6F6RYKHTCYYSA6JA4IOBYEVQDMYGZFII45SVCR5NNB4DC27EKM4A, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.698975928 (791h15m41.161068411s) [noise=true , address=145.79.227.16:4910 , piece_id=KTI55SFT443LQUWK2OB225JXO2G6YS65MKO5SSYKKBLYW7GEYNGA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.724922165 (791h15m41.187014648s) [noise=true , address=145.79.227.2:4113 , piece_id=CA2RCV2Y6A7EQFD54ZD2B7HNZBMO5AYTDE6S346D7IRXGCIRIUIA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.726562328 (791h15m41.188654811s) [noise=true , address=145.79.227.4:4186 , piece_id=TYZFRIMSC7WUASFTG2JP2JKYIE6OQJHZRUWJ4SSPHZ2PTHG2KXPQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.762781125 (791h15m41.224873618s) [noise=true , address=145.79.227.20:4437 , piece_id=PMEFCUCE5KPVZDHYNZYYWXIQCKOZRIVUW2L2NILTYFBG7TMNVAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.770992812 (791h15m41.233085295s) [noise=true , address=145.79.227.22:4509 , piece_id=2GBUPWFX3K53PDZR4RLC22TYR252KE6DBIP2SWSHLT6B5RTH3JKQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.780498642 (791h15m41.242591126s) [noise=true , address=145.79.227.5:4222 , piece_id=PRNNFIZJKJCWAVXNUMUKO4WTBRO2NGEUBL6U66H4DUEVTJAZDVMA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.791784423 (791h15m41.253876896s) [noise=true , address=145.79.227.26:4653 , piece_id=246OFOQF5UGESWQQLVAVXIE2ENUY5I4GO6P3OW7N7EHOVPSIPWFQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.811267893 (791h15m41.273360377s) [noise=true , address=145.79.227.25:4619 , piece_id=VZBKXKAW4XDGSHJHZBD7IRAJPSI5PRI4U4MADSQT3AUYXFGLDAQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:55.937959549 (791h15m41.400052022s) [noise=true , address=77.90.31.19:4843 , piece_id=LZVFRZIOTHF2XUJXS4WXYQDSX6NA4DAKK3FO7JCIDXPCU5H7JJRA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.012483687 (791h15m41.47457616s) [noise=true , address=77.90.31.21:4879 , piece_id=SPUUMWSM44HU4F6RVLX6LH5XY6GXL7DPJU47G2X7SFYUYWU37KMQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034087284 (791h15m41.496179767s) [noise=true , address=77.90.31.15:4697 , piece_id=LVGQRVL4NLZVGSQNVD3ABMX4U4DUN4SINR5W5TGRPEQE5RJXBHQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: <nil> Oct 26 12:46:56.034126839 (791h15m41.496219322s) [seg_pos={0 0}, upload=696025]: Segment reached optimal threshold of 13 pieces. Oct 26 12:46:56.034284791 (791h15m41.496377274s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034292976 (791h15m41.496385459s) [noise=true , address=77.90.31.13:4627 , piece_id=A35YEPHW4TJZ3KEIL2C7OHEGK3A4UISKEAXBRY6C55HBS3FXOZQQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034307042 (791h15m41.496399515s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034317693 (791h15m41.496410176s) [noise=true , address=77.90.31.14:4663 , piece_id=P3E3JCCKRVOHYKK4QBH3DESUW2MV2QM2KD44GESCSFC7QJ72DARQ, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034378919 (791h15m41.496471392s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 26 12:46:56.034386043 (791h15m41.496478526s) [noise=true , address=45.91.116.9:4262 , piece_id=OGMX34KZQLCRZJH2MYX4GWFR7M77BBXPTGQQIFEUENGJ7PDYICQA, seg_pos={0 0}, upload=696025]: Long tail cancellation due to optimal threshold Oct 26 12:46:56.034404238 (791h15m41.496496721s) [seg_pos={0 0}, upload=696025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 26 12:46:56.034520208 (791h15m41.496612691s) [upload=696025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 26 12:46:56.034527112 (791h15m41.496619585s) [upload=696025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 26 12:46:56.358026346 (791h15m41.820118829s) [upload=696025]: Done uploading segments... Oct 26 12:46:56.358080729 (791h15m41.820173202s) [upload=696025]: Upload finished. err: <nil> Duration: 6.27548253s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)