Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X....X........X........................] duration: 7.540338854s FAIL (2025-09-27 18:23:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.540371036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.06022202s FAIL (2025-09-27 18:22:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.060291283s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.027394191s FAIL (2025-09-27 18:22:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.027426681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X....X........X.......................] duration: 6.333906307s FAIL (2025-09-27 18:22:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.333934261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X....X........X......................] duration: 7.215120273s FAIL (2025-09-27 18:21:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.215153407s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.970943494s FAIL (2025-09-27 18:21:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.970995903s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.954060945s FAIL (2025-09-27 18:20:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.954091232s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X....X........X.....................] duration: 6.062702278s FAIL (2025-09-27 18:20:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.062736565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.91006498s FAIL (2025-09-27 18:20:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.9101022s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 7.415300764s FAIL (2025-09-27 18:19:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 7.415329237s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X....X........X....................] duration: 6.140765145s FAIL (2025-09-27 18:19:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.140802206s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X....X........X...................] duration: 6.308262891s FAIL (2025-09-27 18:19:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.308299069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.740522487s FAIL (2025-09-27 18:18:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.740548836s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X....X........X..................] duration: 6.220896509s FAIL (2025-09-27 18:18:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.220940854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.021651631s FAIL (2025-09-27 18:18:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.021684452s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.898470327s FAIL (2025-09-27 18:17:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.898500423s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X....X........X.................] duration: 13.7251369s FAIL (2025-09-27 18:17:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 13.725192886s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.778394313s FAIL (2025-09-27 18:16:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.778423468s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X....X........X................] duration: 6.19760495s FAIL (2025-09-27 18:16:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.197698297s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X....X........X...............] duration: 5.896960187s FAIL (2025-09-27 18:15:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.89698758s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.874036917s FAIL (2025-09-27 18:15:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.874067426s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X....X........X..............] duration: 6.174484214s FAIL (2025-09-27 18:15:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.174515142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X....X........X.............] duration: 6.996190279s FAIL (2025-09-27 18:14:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.996219506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.227588855s FAIL (2025-09-27 18:14:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.22761741s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.923338735s FAIL (2025-09-27 18:14:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.923370555s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X....X........X............] duration: 9.757666634s FAIL (2025-09-27 18:13:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 9.757696549s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X....X........X...........] duration: 6.108401229s FAIL (2025-09-27 18:13:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.108431487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.854073793s FAIL (2025-09-27 18:13:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.854107307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X....X........X..........] duration: 6.437028806s FAIL (2025-09-27 18:12:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.437059916s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.211383844s FAIL (2025-09-27 18:12:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.211417318s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.946367375s FAIL (2025-09-27 18:12:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.946399657s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................X....X........X.........] duration: 6.146861432s FAIL (2025-09-27 18:11:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.146909093s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.109307943s FAIL (2025-09-27 18:11:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.109348631s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X....X........X........] duration: 6.174747004s FAIL (2025-09-27 18:11:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.174780137s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 8.067063796s FAIL (2025-09-27 18:10:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 8.067102148s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................X....X........X.......] duration: 10.349136372s FAIL (2025-09-27 18:10:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 10.349166871s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X....X........X......] duration: 14.134336721s FAIL (2025-09-27 18:10:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 14.134369042s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.087929465s FAIL (2025-09-27 18:09:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 6.087985041s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................X....X........X.....] duration: 6.565975531s FAIL (2025-09-27 18:09:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.566010219s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 9.511687495s FAIL (2025-09-27 18:09:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 9.511716749s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 13.667732014s FAIL (2025-09-27 18:08:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 13.667772391s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X....X........X....] duration: 6.177683216s FAIL (2025-09-27 18:08:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.177716097s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X....X........X...] duration: 6.451835909s FAIL (2025-09-27 18:07:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.451872087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.839362168s FAIL (2025-09-27 18:07:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.839407074s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X....X........X..] duration: 6.30172921s FAIL (2025-09-27 18:07:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.301761962s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 38.088644852s FAIL (2025-09-27 18:07:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 38.088684467s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X....X........X.] duration: 7.303648656s FAIL (2025-09-27 18:06:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.303669026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X....X........X] duration: 30.283075108s FAIL (2025-09-27 18:06:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.283097502s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X....X........] duration: 6.101271782s FAIL (2025-09-27 18:05:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 17:58:51.646030323 (100h27m37.108122806s) [upload=88389]: Starting upload Sep 27 17:58:51.646110787 (100h27m37.10820326s) [upload=88389]: Done starting upload Sep 27 17:58:51.646197001 (100h27m37.108289484s) [upload=88389]: Uploading segments... Sep 27 17:58:51.646464060 (100h27m37.108556533s) [upload=88389]: Got next segment. Inline: false Sep 27 17:58:51.646506320 (100h27m37.108598793s) [upload=88389]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:58:51.646512722 (100h27m37.108605205s) [upload=88389]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:58:51.904123719 (100h27m37.366216202s) [seg_pos={0 0}, upload=88389]: Begin upload segment... Sep 27 17:58:51.904194934 (100h27m37.366287417s) [seg_pos={0 0}, upload=88389]: Done begin upload segment. Sep 27 17:58:51.904222427 (100h27m37.36631491s) [noise=true , address=45.91.116.5:4298 , piece_id=UF3FDM64QYLK6KTURXV6OTBTD4TTOACDPQ4JSSWDLT66DRCOAA3A, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.904237224 (100h27m37.366329697s) [noise=true , address=145.79.227.19:4079 , piece_id=QVBCX5ZAP3ALCO6WBU74JAECKQ4P3IWOCEXLNQYHIETMXVQK7ZZQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.904343056 (100h27m37.366435539s) [noise=true , address=145.79.227.12:4765 , piece_id=IABKUOP36HOHR3CSBFQALWWHD3YBZ7EQZLM2UTPSIKIUDANJZJEA, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.904943861 (100h27m37.367036344s) [noise=true , address=45.91.116.14:4189 , piece_id=CQ3SVNVBG7E2T3RJWLZL7A4LZCBYWOG6ONFYUN4FGG3YC2L4AK2Q, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.905017811 (100h27m37.367110294s) [noise=true , address=145.79.227.11:4732 , piece_id=NVGFQD4HQ3KSXJFL5BBWG254MV7RGMRFZJNUMWDHUPLJI6OK52GQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.905383718 (100h27m37.367476201s) [noise=true , address=145.79.227.19:4078 , piece_id=AD4IZI4B6GJ43Y2OTLZ3YUIXEDWFZYJPOHZ4DNH6U6JGEG4VI7PA, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.905844214 (100h27m37.367936697s) [noise=true , address=45.91.116.9:4262 , piece_id=EB6XG324NNQEBC3BUV6UNNEI6XJ37AJKJSGKM3Y4UOHC5BZIQ4DA, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.906284844 (100h27m37.368377317s) [noise=true , address=77.90.31.12:4590 , piece_id=DISVPQWLXZ2LIS5A2ESME3HSNNPCJVJ6UBEGWIS7AWKDEJ24Y4VQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.906694524 (100h27m37.368787007s) [noise=true , address=145.79.227.17:4970 , piece_id=YBIZZPWQJVEE7W6PGAFDMXO3D3CKSALKWRTUH7HMRSXNCHOZ5CVA, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.907102371 (100h27m37.369194854s) [noise=true , address=77.90.31.15:4697 , piece_id=NVZZXYBYMIEICZFYUDKZIQQ3HVMC2AWERB7KT2LYNUJZCYZGGPWQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.907844123 (100h27m37.369936596s) [noise=true , address=145.79.227.14:4837 , piece_id=O6W4ZTPICDUVA2IJCHKZO3NXOGJT7YDVMBIO43BIDDRASP6X7V7Q, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.908295753 (100h27m37.370388236s) [noise=true , address=77.90.31.13:4626 , piece_id=ZVQMSUBJELPPXSUDTG23ALOYTUSXNEL64M5YNAWNVGYVOKRXT4ZQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.908753945 (100h27m37.370846418s) [noise=true , address=45.91.116.9:4263 , piece_id=G2HWKBGQ6ZRJ6ZGAY5H7HJ6EXAFP46TAR7UVFEWKF3QS3GSDGZZA, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.909215474 (100h27m37.371307947s) [noise=true , address=77.90.31.11:4555 , piece_id=5DXYYTF53GRROLO2S3OIPAEDVU4K42E3GTYKAXACVQYN476DLAUQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.909927861 (100h27m37.372020334s) [noise=true , address=145.79.227.16:4911 , piece_id=24GJGOR4XDP2XH4EVJDIF2KVL622YI5KXR3MAO2POR7527EYA4DQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:51.911821667 (100h27m37.37391415s) [noise=true , address=45.91.116.13:4153 , piece_id=XIFAR652JL7XUJK26ATL4JUQXE4NAL4M5EQBGDZ34DS62PH5H2XQ, seg_pos={0 0}, upload=88389]: Uploading piece... Sep 27 17:58:53.644536448 (100h27m39.106628931s) [upload=88389]: Next returned nil segment Sep 27 17:58:53.644560625 (100h27m39.106653098s) [upload=88389]: Waiting for error group managing segments... Sep 27 17:58:53.934428424 (100h27m39.396520907s) [noise=true , address=145.79.227.12:4765 , piece_id=IABKUOP36HOHR3CSBFQALWWHD3YBZ7EQZLM2UTPSIKIUDANJZJEA, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.940709835 (100h27m39.402802318s) [noise=true , address=145.79.227.17:4970 , piece_id=YBIZZPWQJVEE7W6PGAFDMXO3D3CKSALKWRTUH7HMRSXNCHOZ5CVA, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.941616311 (100h27m39.403708794s) [noise=true , address=145.79.227.19:4079 , piece_id=QVBCX5ZAP3ALCO6WBU74JAECKQ4P3IWOCEXLNQYHIETMXVQK7ZZQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.945155079 (100h27m39.407247562s) [noise=true , address=145.79.227.19:4078 , piece_id=AD4IZI4B6GJ43Y2OTLZ3YUIXEDWFZYJPOHZ4DNH6U6JGEG4VI7PA, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.948692025 (100h27m39.410784508s) [noise=true , address=145.79.227.14:4837 , piece_id=O6W4ZTPICDUVA2IJCHKZO3NXOGJT7YDVMBIO43BIDDRASP6X7V7Q, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.949963896 (100h27m39.412056369s) [noise=true , address=145.79.227.11:4732 , piece_id=NVGFQD4HQ3KSXJFL5BBWG254MV7RGMRFZJNUMWDHUPLJI6OK52GQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:53.953847331 (100h27m39.415939824s) [noise=true , address=145.79.227.16:4911 , piece_id=24GJGOR4XDP2XH4EVJDIF2KVL622YI5KXR3MAO2POR7527EYA4DQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:54.012501789 (100h27m39.474594263s) [noise=true , address=45.91.116.9:4262 , piece_id=EB6XG324NNQEBC3BUV6UNNEI6XJ37AJKJSGKM3Y4UOHC5BZIQ4DA, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:54.017954321 (100h27m39.480046804s) [noise=true , address=45.91.116.14:4189 , piece_id=CQ3SVNVBG7E2T3RJWLZL7A4LZCBYWOG6ONFYUN4FGG3YC2L4AK2Q, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:54.076293660 (100h27m39.538386143s) [noise=true , address=45.91.116.13:4153 , piece_id=XIFAR652JL7XUJK26ATL4JUQXE4NAL4M5EQBGDZ34DS62PH5H2XQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:54.099823755 (100h27m39.561916238s) [noise=true , address=45.91.116.9:4263 , piece_id=G2HWKBGQ6ZRJ6ZGAY5H7HJ6EXAFP46TAR7UVFEWKF3QS3GSDGZZA, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:58:54.118721361 (100h27m39.580813834s) [noise=true , address=45.91.116.5:4298 , piece_id=UF3FDM64QYLK6KTURXV6OTBTD4TTOACDPQ4JSSWDLT66DRCOAA3A, seg_pos={0 0}, upload=88389]: Done uploading piece. err: <nil> Sep 27 17:59:20.923660523 (100h28m6.385752996s) [noise=true , address=77.90.31.12:4590 , piece_id=DISVPQWLXZ2LIS5A2ESME3HSNNPCJVJ6UBEGWIS7AWKDEJ24Y4VQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:50402->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:59:20.923738842 (100h28m6.385831325s) [noise=true , address=77.90.31.11:4555 , piece_id=5DXYYTF53GRROLO2S3OIPAEDVU4K42E3GTYKAXACVQYN476DLAUQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:55844->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:59:20.923798646 (100h28m6.385891119s) [noise=true , address=77.90.31.13:4626 , piece_id=ZVQMSUBJELPPXSUDTG23ALOYTUSXNEL64M5YNAWNVGYVOKRXT4ZQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: context deadline exceeded; context deadline exceeded Sep 27 17:59:20.923831678 (100h28m6.385924161s) [noise=true , address=77.90.31.15:4697 , piece_id=NVZZXYBYMIEICZFYUDKZIQQ3HVMC2AWERB7KT2LYNUJZCYZGGPWQ, seg_pos={0 0}, upload=88389]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:57208->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:59:20.923870252 (100h28m6.385962725s) [seg_pos={0 0}, upload=88389]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 17:59:21.179223740 (100h28m6.641316223s) [upload=88389]: Done uploading segments... Sep 27 17:59:21.179253145 (100h28m6.641345618s) [upload=88389]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.101300786s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 11.513646477s FAIL (2025-09-27 18:05:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 11.513717835s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)