Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 7.643561845s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:18:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.643583447s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 8.501779808s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:18:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.501804257s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.519236769s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:17:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.519254944s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 7.254014224s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:16:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.254045114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 11.309819483s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:16:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 11.309841775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 11.015937659s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:15:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 11.015963117s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.723364273s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:14:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.723389051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 7.447621472s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:14:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.447648353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.890653371s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:14:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.890682437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 8.304252891s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:14:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.304271277s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 8.635780409s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:13:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.635796722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.709874644s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:13:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.70989829s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 8.674246922s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:12:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.674261299s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.734085547s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:12:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.734111245s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.750709549s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:11:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.750736933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 8.432607698s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:11:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.432631465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 7.538119117s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:11:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.538138895s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.366107135s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:11:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 6.366132194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 19.045695518s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:10:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 19.045722159s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.737050106s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:10:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.737076849s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 3.467944839s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:09:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.467966912s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 18.756447983s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:09:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 18.756479203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.725207202s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:08:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.725234124s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 3.436972205s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:08:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.436986202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 8.131934613s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:08:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.131961195s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 3.491822953s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:08:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.491844493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.755329816s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:08:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.75534664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.88078221s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:07:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.880806747s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.688923103s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:07:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.688948201s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 3.371659745s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:07:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.371684402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 10.046296137s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:07:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 10.046319531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.853371079s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:07:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.853391428s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 3.411806457s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:06:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.411838048s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.348169749s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:05:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 6.348193985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.808225634s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:05:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.808381574s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 4.40478486s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:05:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.404807353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.854335896s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:04:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.854357909s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 7.074962127s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:04:43 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.074990803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.770895385s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:04:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.770920334s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 3.427387368s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:04:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.42741479s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.825675555s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:04:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.825695202s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 3.422154058s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:04:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.422179807s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.792147724s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:03:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.792176459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.736947671s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:03:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.73697292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 3.510411498s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:02:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.510436678s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.261127687s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:02:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 7.261153407s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 4.340369261s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:02:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.340393117s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 3.401303492s trace: https://jaeger.dev.storj.io/trace/3ee8b64a70516f0c FAIL (2026-04-08 06:02:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.40132837s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 8.548599021s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:01:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.548623157s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.792779602s trace: https://jaeger.dev.storj.io/trace/12f2c2dc9f7f7ae5 FAIL (2026-04-08 06:01:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.82897888s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 8 05:28:25.009748667 (13h42m11.828984702s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 8 05:28:25.248005204 (13h42m12.067241228s) [seg_pos={0 0}, upload=13157]: Begin upload segment... Apr 8 05:28:25.248060962 (13h42m12.067296996s) [seg_pos={0 0}, upload=13157]: Done begin upload segment. Apr 8 05:28:25.248107552 (13h42m12.067343576s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248139102 (13h42m12.067375126s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248921332 (13h42m12.068157366s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.248946200 (13h42m12.068182244s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249267761 (13h42m12.068503795s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249347344 (13h42m12.068583368s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.249596686 (13h42m12.06883271s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250052416 (13h42m12.06928844s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250452898 (13h42m12.069688922s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.250779799 (13h42m12.070015833s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251092963 (13h42m12.070328987s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251487775 (13h42m12.070723809s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.251926252 (13h42m12.071162286s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253102752 (13h42m12.072338776s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.253485942 (13h42m12.072721966s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:25.254722949 (13h42m12.073958983s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Uploading piece... Apr 8 05:28:27.038321881 (13h42m13.857557905s) [upload=13157]: Next returned nil segment Apr 8 05:28:27.038334716 (13h42m13.85757074s) [upload=13157]: Waiting for error group managing segments... Apr 8 05:28:27.333568043 (13h42m14.152804068s) [noise=true , address=145.79.227.10:4405 , piece_id=BJ4OCWYC6A4VPCTFWFWBGCFXBRY5IIG2BVIQEJ2SL77OJAKZLMMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.335178452 (13h42m14.154414476s) [noise=true , address=145.79.227.2:4117 , piece_id=EI64LGHZHEN6RPIGMLOO737JLZ6KPBSTN2F2JJZPPPFIVC7G5MMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.339949740 (13h42m14.159185764s) [noise=true , address=145.79.227.44:4195 , piece_id=ZOXW6MDONW7LCF4OYBPBI2HYA4K7YTDKWRIL6674LBKCSPCYXKXA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.340699567 (13h42m14.159935581s) [noise=true , address=145.79.227.41:4086 , piece_id=JDR372AAP4FGLOCWNC7HVZZUWKAHJRBBRRI3TFMOL7CH23ROTC5Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.343731748 (13h42m14.162967773s) [noise=true , address=145.79.227.42:4124 , piece_id=3YBAMZGBF4FA5JXA7JYDAHKIYMXFXOOTLEOFZ2BPV6ZFQ246KE3A, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.347652345 (13h42m14.166888369s) [noise=true , address=145.79.227.19:4083 , piece_id=P57JQZZJXTXEDJ4QUMK2JROTRPI3GEZUD42INTIOGJOV6IMMSRWA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.352712661 (13h42m14.171948695s) [noise=true , address=77.90.31.16:4738 , piece_id=HB54HT5UIWBYRSJS6PZFD7U4AK57WPKSWOPRHPNPHJSPG6WVCFRQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.355753109 (13h42m14.174989133s) [noise=true , address=77.90.31.18:4808 , piece_id=B2IOX6NIFGOP3E4DBBO36ZTQHX72UXZGNTB7LBEZJE2L37XRSPMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.365858592 (13h42m14.185094616s) [noise=true , address=145.79.227.27:4694 , piece_id=UPA6Z24ODQWDRQ3OQFFNFS5U3O7CIV7HYHILEBRIYYKIIDGDSHMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.372758488 (13h42m14.191994512s) [noise=true , address=145.79.227.40:4049 , piece_id=OAEPXKTP35ISWJIMRP2FEMHPLCBB6TLB6Z3STUNAH65KURSMM4SQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.373548082 (13h42m14.192784106s) [noise=true , address=145.79.227.3:4155 , piece_id=BNT3VSM427P76IVXEUENVQJOTSPDCAO6N24CPN4QBPHXJ46JJW6Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.385745403 (13h42m14.204981427s) [noise=true , address=145.79.227.36:4517 , piece_id=GB7GJJ6CM4NSXH5VJMZLV4XXUVEAJRXOQDDUFWAUYLDVQZGXDEXQ, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419293187 (13h42m14.238529211s) [noise=true , address=145.79.227.26:4656 , piece_id=BQ2I6AESNFKJ43TINET4KM56D3XCP3BPMETSNYGJ2GPYQYJRAX3Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: <nil> Apr 8 05:28:27.419318536 (13h42m14.23855455s) [seg_pos={0 0}, upload=13157]: Satisfied cohort requirements after 13 pieces Apr 8 05:28:27.419506839 (13h42m14.238742853s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo): protocol: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37070->45.91.116.8:4070: use of closed network connection Apr 8 05:28:27.419513112 (13h42m14.238749136s) [noise=true , address=45.91.116.8:4070 , piece_id=YPY2FL7J446ZSPGW667CGSOOZP43DEHSF2DOSYF6TKPDZTAO3GMA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419767082 (13h42m14.239003106s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46466->45.91.116.5:4297: use of closed network connection Apr 8 05:28:27.419773454 (13h42m14.239009478s) [noise=true , address=45.91.116.5:4297 , piece_id=EGNF4A7QEDCFZC6T4OMQ4KA36K2ZLEJL2L7YNJKRAE44I7HZE2WA, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419868988 (13h42m14.239105032s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT): protocol: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41576->45.91.116.4:4953: use of closed network connection Apr 8 05:28:27.419879479 (13h42m14.239115503s) [noise=true , address=45.91.116.4:4953 , piece_id=NDRT54IXUUCRQSBX72CBL3AXLUEDYYNLOHZ4W4OHYYL7DG2IHJ4Q, seg_pos={0 0}, upload=13157]: Long tail cancellation due to optimal threshold Apr 8 05:28:27.419911360 (13h42m14.239147384s) [seg_pos={0 0}, upload=13157]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 8 05:28:27.419973420 (13h42m14.239209444s) [upload=13157]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:28:27.419978099 (13h42m14.239214123s) [upload=13157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:28:27.699112587 (13h42m14.518348611s) [upload=13157]: Done uploading segments... Apr 8 05:28:27.699130241 (13h42m14.518366265s) [upload=13157]: Upload finished. err: <nil> Duration: 5.792804069s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)