Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 902488 reports (Page 11309 of 18050)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
31 out of 60 is failed (threshold=3) [.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X.....XX]
duration: 
30.003542608s
trace: 
https://jaeger.dev.storj.io/trace/18f948c5b083b8c4
FAIL (2025-12-29 04:32:39 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: OFMBOQRNZKX2EJBCBVP2Z6IEVKFIK4SN2JWKSM4PPUOLWCXNWIIA) context deadline exceeded; piecestore: (Node ID: 1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9, Piece ID: 3TYKVCO53VF56WUITAIGGTZZJC6K7RJUGU6IK7HNHSFO7ZZ6F45A) context deadline exceeded
======LOGS======
Dec 29 04:32:09.992197456 (565h3m9.863280501s) [upload=495415]: Starting upload
Dec 29 04:32:09.992253463 (565h3m9.863336498s) [upload=495415]: Done starting upload
Dec 29 04:32:09.992301013 (565h3m9.863384048s) [upload=495415]: Uploading segments...
Dec 29 04:32:09.992434657 (565h3m9.863517702s) [upload=495415]: Got next segment. Inline: false
Dec 29 04:32:09.992478420 (565h3m9.863561456s) [upload=495415]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:32:09.992482789 (565h3m9.863565834s) [upload=495415]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:32:10.240234822 (565h3m10.111317857s) [seg_pos={0 0}, upload=495415]: Begin upload segment...
Dec 29 04:32:10.240294356 (565h3m10.111377401s) [seg_pos={0 0}, upload=495415]: Done begin upload segment.
Dec 29 04:32:10.240394216 (565h3m10.111477261s) [noise=true , address=77.90.31.10:4519     , piece_id=OFMBOQRNZKX2EJBCBVP2Z6IEVKFIK4SN2JWKSM4PPUOLWCXNWIIA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.240400107 (565h3m10.111483142s) [noise=true , address=145.79.227.18:4006   , piece_id=2FSAB5BTGPRMTNFWHODW7ZCX3R6G7V6OEYEYBXGAAWRDYWRCJ6XQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.241098026 (565h3m10.112181071s) [noise=true , address=145.79.227.51:4411   , piece_id=NS4KDVEAOOMYIWXOUXB5PKJDK5E3BKQFVNGOOXZELEEGFHDMKXHA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.241496165 (565h3m10.1125792s) [noise=true , address=145.79.227.39:4015   , piece_id=NLRWDL7NKEMIPNAMQO2L3KYH4SCC5ZWWYGTZLOTCY5RQF67FWVMA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.241666719 (565h3m10.112749764s) [noise=true , address=45.91.116.7:4009     , piece_id=HOXOVXQUEPJ5RPX4OPK2IDDQDCJMIKY23JB5XLBB4GOSKP6HFQLQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.241844789 (565h3m10.112927834s) [noise=true , address=145.79.227.20:4438   , piece_id=MFONCVINAMKTXDLB7H4XONXT7VAFNNE2JCY4RA5RIRJTMOT3WXSQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.242131765 (565h3m10.11321481s) [noise=true , address=45.91.116.12:4226    , piece_id=TGRBKEJMM53CL4PLJXFFSCURTDB7TQPBMI2V4BDGDFQABWSBNS4A, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.242277883 (565h3m10.113360928s) [noise=true , address=145.79.227.2:4115    , piece_id=GY37TIZXLFHONVFCK3OFZ542D63P5ZJ5IW6MC5JOTEVI5KV6FQRA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.242408402 (565h3m10.113491447s) [noise=true , address=145.79.227.47:4265   , piece_id=EZTBVATUYSH2C7VBKNA3JASDFHHMMFQVHAOX6UYETGY7ZVLDA5NA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.242741616 (565h3m10.113824661s) [noise=true , address=145.79.227.45:4231   , piece_id=VXZARYHCSPTWRCKCS3KKQIMLAD3YJ7BD7FBQ7BU743GDJWU5QJIQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.242778586 (565h3m10.113861631s) [noise=true , address=145.79.227.26:4654   , piece_id=Y4DJBEVUBNPM56MWRENRRNCPKYAOH4PGP4LGG5DXVWJKVLZNQXLQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.243054521 (565h3m10.114137556s) [noise=true , address=77.90.31.11:4555     , piece_id=W6KNN25UWDD44PECSMPTA5ONUK2F5BAX7BTDZWREZASS44YTCGDA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.243263630 (565h3m10.114346675s) [noise=true , address=45.91.116.14:4191    , piece_id=ISKOCJYDUCRZA2WQW57HUAY2ZHH5V7YA5ADGXNHQ5AH54ER2IP2A, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.243738184 (565h3m10.114821229s) [noise=true , address=145.79.227.14:4840   , piece_id=LBWFNHCSL7TN3NSFFPKNX3MTRREJO4FGVSSUJ32OFRFQL6YTN2OQ, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.244208699 (565h3m10.115291744s) [noise=true , address=77.90.31.18:4808     , piece_id=3TYKVCO53VF56WUITAIGGTZZJC6K7RJUGU6IK7HNHSFO7ZZ6F45A, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:10.244688813 (565h3m10.115771859s) [noise=true , address=77.90.31.16:4736     , piece_id=HMDSRFELYGANZ4LJ554ETRQPBXEFKTQYZIUGG5H7GDW2N5VZJ3AA, seg_pos={0 0}, upload=495415]: Uploading piece...
Dec 29 04:32:11.846904045 (565h3m11.717987081s) [upload=495415]: Next returned nil segment
Dec 29 04:32:11.846918103 (565h3m11.718001148s) [upload=495415]: Waiting for error group managing segments...
Dec 29 04:32:12.229872710 (565h3m12.100955755s) [noise=true , address=145.79.227.39:4015   , piece_id=NLRWDL7NKEMIPNAMQO2L3KYH4SCC5ZWWYGTZLOTCY5RQF67FWVMA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.230305845 (565h3m12.10138889s) [noise=true , address=145.79.227.18:4006   , piece_id=2FSAB5BTGPRMTNFWHODW7ZCX3R6G7V6OEYEYBXGAAWRDYWRCJ6XQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.230448627 (565h3m12.101531662s) [noise=true , address=145.79.227.14:4840   , piece_id=LBWFNHCSL7TN3NSFFPKNX3MTRREJO4FGVSSUJ32OFRFQL6YTN2OQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.230920104 (565h3m12.102003149s) [noise=true , address=145.79.227.2:4115    , piece_id=GY37TIZXLFHONVFCK3OFZ542D63P5ZJ5IW6MC5JOTEVI5KV6FQRA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.232893181 (565h3m12.103976226s) [noise=true , address=145.79.227.20:4438   , piece_id=MFONCVINAMKTXDLB7H4XONXT7VAFNNE2JCY4RA5RIRJTMOT3WXSQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.244267639 (565h3m12.115350685s) [noise=true , address=145.79.227.51:4411   , piece_id=NS4KDVEAOOMYIWXOUXB5PKJDK5E3BKQFVNGOOXZELEEGFHDMKXHA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.248074037 (565h3m12.119157082s) [noise=true , address=145.79.227.26:4654   , piece_id=Y4DJBEVUBNPM56MWRENRRNCPKYAOH4PGP4LGG5DXVWJKVLZNQXLQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.249718279 (565h3m12.120801324s) [noise=true , address=145.79.227.45:4231   , piece_id=VXZARYHCSPTWRCKCS3KKQIMLAD3YJ7BD7FBQ7BU743GDJWU5QJIQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.253244062 (565h3m12.124327097s) [noise=true , address=145.79.227.47:4265   , piece_id=EZTBVATUYSH2C7VBKNA3JASDFHHMMFQVHAOX6UYETGY7ZVLDA5NA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.530094292 (565h3m12.401177337s) [noise=true , address=77.90.31.10:4519     , piece_id=OFMBOQRNZKX2EJBCBVP2Z6IEVKFIK4SN2JWKSM4PPUOLWCXNWIIA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.575896035 (565h3m12.44697908s) [noise=true , address=77.90.31.16:4736     , piece_id=HMDSRFELYGANZ4LJ554ETRQPBXEFKTQYZIUGG5H7GDW2N5VZJ3AA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:12.589686152 (565h3m12.460769187s) [noise=true , address=45.91.116.12:4226    , piece_id=TGRBKEJMM53CL4PLJXFFSCURTDB7TQPBMI2V4BDGDFQABWSBNS4A, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:13.167908535 (565h3m13.03899157s) [noise=true , address=77.90.31.18:4808     , piece_id=3TYKVCO53VF56WUITAIGGTZZJC6K7RJUGU6IK7HNHSFO7ZZ6F45A, seg_pos={0 0}, upload=495415]: Done uploading piece. err: <nil>
Dec 29 04:32:13.167933092 (565h3m13.039016137s) [seg_pos={0 0}, upload=495415]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:32:13.168168521 (565h3m13.039251576s) [noise=true , address=45.91.116.7:4009     , piece_id=HOXOVXQUEPJ5RPX4OPK2IDDQDCJMIKY23JB5XLBB4GOSKP6HFQLQ, seg_pos={0 0}, upload=495415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF
Dec 29 04:32:13.168181144 (565h3m13.039264199s) [noise=true , address=45.91.116.7:4009     , piece_id=HOXOVXQUEPJ5RPX4OPK2IDDQDCJMIKY23JB5XLBB4GOSKP6HFQLQ, seg_pos={0 0}, upload=495415]: Long tail cancellation due to optimal threshold
Dec 29 04:32:13.168263791 (565h3m13.039346826s) [noise=true , address=45.91.116.14:4191    , piece_id=ISKOCJYDUCRZA2WQW57HUAY2ZHH5V7YA5ADGXNHQ5AH54ER2IP2A, seg_pos={0 0}, upload=495415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:32:13.168279672 (565h3m13.039362707s) [noise=true , address=45.91.116.14:4191    , piece_id=ISKOCJYDUCRZA2WQW57HUAY2ZHH5V7YA5ADGXNHQ5AH54ER2IP2A, seg_pos={0 0}, upload=495415]: Long tail cancellation due to optimal threshold
Dec 29 04:32:13.168449355 (565h3m13.03953239s) [noise=true , address=77.90.31.11:4555     , piece_id=W6KNN25UWDD44PECSMPTA5ONUK2F5BAX7BTDZWREZASS44YTCGDA, seg_pos={0 0}, upload=495415]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: write tcp 188.166.225.170:50674->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:50674->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50674->77.90.31.11:4555: use of closed network connection
Dec 29 04:32:13.168456238 (565h3m13.039539283s) [noise=true , address=77.90.31.11:4555     , piece_id=W6KNN25UWDD44PECSMPTA5ONUK2F5BAX7BTDZWREZASS44YTCGDA, seg_pos={0 0}, upload=495415]: Long tail cancellation due to optimal threshold
Dec 29 04:32:13.168471136 (565h3m13.039554171s) [seg_pos={0 0}, upload=495415]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:32:13.168564865 (565h3m13.0396479s) [upload=495415]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:32:13.168569634 (565h3m13.039652669s) [upload=495415]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:32:13.462022598 (565h3m13.333105653s) [upload=495415]: Done uploading segments...
Dec 29 04:32:13.462050231 (565h3m13.333133266s) [upload=495415]: Upload finished. err: <nil>
Duration: 30.003560843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [...X.X......X.............X.................................]
duration: 
30.10050115s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:32:36 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 30.100518203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
31 out of 60 is failed (threshold=3) [X.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X.....X]
duration: 
30.005523467s
trace: 
https://jaeger.dev.storj.io/trace/10f620b1699c55d7
FAIL (2025-12-29 04:32:09 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: PZONTOBLHTP2F3DQIS7RMIWPPBUQKMEHJ5ZY3TVMVHRBZUZVFVRQ) context deadline exceeded; piecestore: (Node ID: 12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, Piece ID: 72FP2O3YUICYUFTDSANXILVYJH5J5TNDAPTIT2AAMOPGRPDFKFMQ) context deadline exceeded
======LOGS======
Dec 29 04:31:39.757369714 (565h2m39.628452749s) [upload=495408]: Starting upload
Dec 29 04:31:39.757426702 (565h2m39.628509748s) [upload=495408]: Done starting upload
Dec 29 04:31:39.757546089 (565h2m39.628629134s) [upload=495408]: Uploading segments...
Dec 29 04:31:39.757635490 (565h2m39.628718545s) [upload=495408]: Got next segment. Inline: false
Dec 29 04:31:39.757695845 (565h2m39.62877889s) [upload=495408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:31:39.757703059 (565h2m39.628786104s) [upload=495408]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:31:40.000611378 (565h2m39.871694433s) [seg_pos={0 0}, upload=495408]: Begin upload segment...
Dec 29 04:31:40.000669449 (565h2m39.871752484s) [seg_pos={0 0}, upload=495408]: Done begin upload segment.
Dec 29 04:31:40.000700599 (565h2m39.871783654s) [noise=true , address=45.91.116.5:4297     , piece_id=PZONTOBLHTP2F3DQIS7RMIWPPBUQKMEHJ5ZY3TVMVHRBZUZVFVRQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.000710547 (565h2m39.871793582s) [noise=true , address=45.91.116.6:4976     , piece_id=3Z2YIIJZXC2ECNQPLDVLVZLXXYQ4IYUZNSHUH6PS7JOVBIOHR3HA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.000778757 (565h2m39.871861832s) [noise=true , address=45.91.116.8:4047     , piece_id=72FP2O3YUICYUFTDSANXILVYJH5J5TNDAPTIT2AAMOPGRPDFKFMQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.001421290 (565h2m39.872504335s) [noise=true , address=77.90.31.13:4626     , piece_id=4KCHOU3FXPLLP6XDJJIF3Z4IQWGCQCFKWZSCDYEDC2SALYKAAZSQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.001683219 (565h2m39.872766254s) [noise=true , address=45.91.116.11:4117    , piece_id=JZMF6UNPEEJAMNGDKV47B33YVOAKECFIW56BMC24JVCFTJXGCVBA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.001855307 (565h2m39.872938352s) [noise=true , address=145.79.227.3:4150    , piece_id=7UUF2YS4UEYIYDSOTYNNHKQ3QRUMO4CORQMX6KHKTOPGGOEPDCSA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.002099722 (565h2m39.873182768s) [noise=true , address=145.79.227.11:4731   , piece_id=WX7DVKSKC33BBGZUXYVV5ROG7IC3I4GKQIQ4E5ZLRE7V3S5UA6ZQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.002204762 (565h2m39.873287798s) [noise=true , address=145.79.227.34:4907   , piece_id=XT37G5K4KQ577WWIBIA2QDCFPTU2OV2UNSHPADLRCQTHSCP6PQ2Q, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.002705175 (565h2m39.87378821s) [noise=true , address=145.79.227.15:4873   , piece_id=OX5VNYTH5BQSHKV37ZDI4Y6MCHMVOJ42F54K5YAL3S2XIPNQUH4A, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.003113533 (565h2m39.874196568s) [noise=true , address=145.79.227.17:4971   , piece_id=NGWDKQH76JTGXT374AMA6C4L5PD25U7XEKEXZF3RVDLWHDT6E5YQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.003605710 (565h2m39.874688755s) [noise=true , address=145.79.227.37:4566   , piece_id=SXJEYXE2RDAVK763MUY4ND32DJFGBOH7HYBQPSE4TYLYEYH3EMOA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.003963210 (565h2m39.875046245s) [noise=true , address=45.91.116.4:4940     , piece_id=P33AUTVYELWTXJMXHXG3Q2GRI4RVLCIZTMSI2WNNKWBP2BQC6TCA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.004312597 (565h2m39.875395642s) [noise=true , address=145.79.227.18:4007   , piece_id=THPGUMFKQRSCTBUJBHXYJVOHJAIUC6MNT4I4TDITBS4VZRZZQVQA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.004675217 (565h2m39.875758262s) [noise=true , address=45.91.116.10:4083    , piece_id=SVZKRCUKRRU3Q3PKBG2UOAH5NZYNM7N5FP3ZFZ7QRLDANUHI74IA, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.005046494 (565h2m39.876129539s) [noise=true , address=145.79.227.50:4373   , piece_id=QTSGLWCSQHXR7XSENJXS6U6GMVY4A4BORL2RX3ZHCFWFMWLMYADQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:40.005192081 (565h2m39.876275136s) [noise=true , address=145.79.227.38:4977   , piece_id=BQJCJVSD2RYUVJOAM64AIOB7DH3RIJPBBONRQ3TYSJMM3PDPD7NQ, seg_pos={0 0}, upload=495408]: Uploading piece...
Dec 29 04:31:41.604212229 (565h2m41.475295264s) [upload=495408]: Next returned nil segment
Dec 29 04:31:41.604223841 (565h2m41.475306876s) [upload=495408]: Waiting for error group managing segments...
Dec 29 04:31:41.986890860 (565h2m41.857973895s) [noise=true , address=145.79.227.15:4873   , piece_id=OX5VNYTH5BQSHKV37ZDI4Y6MCHMVOJ42F54K5YAL3S2XIPNQUH4A, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:41.987079229 (565h2m41.858162274s) [noise=true , address=145.79.227.50:4373   , piece_id=QTSGLWCSQHXR7XSENJXS6U6GMVY4A4BORL2RX3ZHCFWFMWLMYADQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:41.992130888 (565h2m41.863213923s) [noise=true , address=145.79.227.37:4566   , piece_id=SXJEYXE2RDAVK763MUY4ND32DJFGBOH7HYBQPSE4TYLYEYH3EMOA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:41.998874929 (565h2m41.869957975s) [noise=true , address=145.79.227.18:4007   , piece_id=THPGUMFKQRSCTBUJBHXYJVOHJAIUC6MNT4I4TDITBS4VZRZZQVQA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.004634907 (565h2m41.875717962s) [noise=true , address=145.79.227.11:4731   , piece_id=WX7DVKSKC33BBGZUXYVV5ROG7IC3I4GKQIQ4E5ZLRE7V3S5UA6ZQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.005042063 (565h2m41.876125108s) [noise=true , address=145.79.227.3:4150    , piece_id=7UUF2YS4UEYIYDSOTYNNHKQ3QRUMO4CORQMX6KHKTOPGGOEPDCSA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.006100908 (565h2m41.877183943s) [noise=true , address=145.79.227.17:4971   , piece_id=NGWDKQH76JTGXT374AMA6C4L5PD25U7XEKEXZF3RVDLWHDT6E5YQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.015423941 (565h2m41.886506976s) [noise=true , address=145.79.227.38:4977   , piece_id=BQJCJVSD2RYUVJOAM64AIOB7DH3RIJPBBONRQ3TYSJMM3PDPD7NQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.325628091 (565h2m42.196711126s) [noise=true , address=45.91.116.8:4047     , piece_id=72FP2O3YUICYUFTDSANXILVYJH5J5TNDAPTIT2AAMOPGRPDFKFMQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.331534287 (565h2m42.202617322s) [noise=true , address=45.91.116.11:4117    , piece_id=JZMF6UNPEEJAMNGDKV47B33YVOAKECFIW56BMC24JVCFTJXGCVBA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.363090305 (565h2m42.23417335s) [noise=true , address=45.91.116.10:4083    , piece_id=SVZKRCUKRRU3Q3PKBG2UOAH5NZYNM7N5FP3ZFZ7QRLDANUHI74IA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.370117445 (565h2m42.24120049s) [noise=true , address=45.91.116.4:4940     , piece_id=P33AUTVYELWTXJMXHXG3Q2GRI4RVLCIZTMSI2WNNKWBP2BQC6TCA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.371658098 (565h2m42.242741133s) [noise=true , address=45.91.116.5:4297     , piece_id=PZONTOBLHTP2F3DQIS7RMIWPPBUQKMEHJ5ZY3TVMVHRBZUZVFVRQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: <nil>
Dec 29 04:31:42.371671775 (565h2m42.24275481s) [seg_pos={0 0}, upload=495408]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:31:42.371904096 (565h2m42.242987132s) [noise=true , address=145.79.227.34:4907   , piece_id=XT37G5K4KQ577WWIBIA2QDCFPTU2OV2UNSHPADLRCQTHSCP6PQ2Q, seg_pos={0 0}, upload=495408]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Dec 29 04:31:42.371911912 (565h2m42.242994947s) [noise=true , address=145.79.227.34:4907   , piece_id=XT37G5K4KQ577WWIBIA2QDCFPTU2OV2UNSHPADLRCQTHSCP6PQ2Q, seg_pos={0 0}, upload=495408]: Long tail cancellation due to optimal threshold
Dec 29 04:31:42.372169232 (565h2m42.243252277s) [noise=true , address=45.91.116.6:4976     , piece_id=3Z2YIIJZXC2ECNQPLDVLVZLXXYQ4IYUZNSHUH6PS7JOVBIOHR3HA, seg_pos={0 0}, upload=495408]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp): protocol: expected piece hash; context canceled; EOF
Dec 29 04:31:42.372176275 (565h2m42.24325931s) [noise=true , address=45.91.116.6:4976     , piece_id=3Z2YIIJZXC2ECNQPLDVLVZLXXYQ4IYUZNSHUH6PS7JOVBIOHR3HA, seg_pos={0 0}, upload=495408]: Long tail cancellation due to optimal threshold
Dec 29 04:31:42.372296725 (565h2m42.24337976s) [noise=true , address=77.90.31.13:4626     , piece_id=4KCHOU3FXPLLP6XDJJIF3Z4IQWGCQCFKWZSCDYEDC2SALYKAAZSQ, seg_pos={0 0}, upload=495408]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Dec 29 04:31:42.372302175 (565h2m42.24338521s) [noise=true , address=77.90.31.13:4626     , piece_id=4KCHOU3FXPLLP6XDJJIF3Z4IQWGCQCFKWZSCDYEDC2SALYKAAZSQ, seg_pos={0 0}, upload=495408]: Long tail cancellation due to optimal threshold
Dec 29 04:31:42.372316913 (565h2m42.243399958s) [seg_pos={0 0}, upload=495408]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:31:42.372376086 (565h2m42.243459121s) [upload=495408]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:31:42.372380865 (565h2m42.2434639s) [upload=495408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:31:42.658144777 (565h2m42.529227812s) [upload=495408]: Done uploading segments...
Dec 29 04:31:42.658162281 (565h2m42.529245316s) [upload=495408]: Upload finished. err: <nil>
Duration: 30.005552803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [....X.X......X.............X................................]
duration: 
38.585383935s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:32:05 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 38.585410564s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [.X.................XX..X.XXXX..X...X.........XX.......X.....]
duration: 
3.464467298s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:31:48 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.464491865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
31 out of 60 is failed (threshold=3) [XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X.....]
duration: 
6.05255305s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:30:59 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 6.052579078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [.....X.X......X.............X...............................]
duration: 
24.908075853s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:30:53 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 24.908094147s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [..X.................XX..X.XXXX..X...X.........XX.......X....]
duration: 
3.901357265s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:30:51 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.901379848s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
31 out of 60 is failed (threshold=3) [.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X....]
duration: 
5.833231604s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:30:28 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 5.833264588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [......X.X......X.............X..............................]
duration: 
52.553080026s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:30:28 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 52.553113038s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [...X.................XX..X.XXXX..X...X.........XX.......X...]
duration: 
3.88379728s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:30:13 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.88382855s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [....X.................XX..X.XXXX..X...X.........XX.......X..]
duration: 
4.432347313s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:29:49 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.432378973s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
32 out of 60 is failed (threshold=3) [X.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X...]
duration: 
25.635858127s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:29:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 25.635888436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [.......X.X......X.............X.............................]
duration: 
47.573029328s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:29:31 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 47.573078701s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [.....X.................XX..X.XXXX..X...X.........XX.......X.]
duration: 
3.408295441s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:28:46 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.408325388s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
33 out of 60 is failed (threshold=3) [XX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X..]
duration: 
5.922616936s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:28:19 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 5.922646032s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [......X.................XX..X.XXXX..X...X.........XX.......X]
duration: 
5.001780155s
trace: 
https://jaeger.dev.storj.io/trace/512a1ea1d4a4e39
FAIL (2025-12-29 04:28:13 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001817476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [X.......X.X......X.............X............................]
duration: 
42.424964187s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:28:12 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 42.424991269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
34 out of 60 is failed (threshold=3) [XXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X.]
duration: 
5.673645424s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:27:35 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 5.673673015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [XXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.X]
duration: 
30.002236602s
trace: 
https://jaeger.dev.storj.io/trace/5b8bd21a3ec16e0c
FAIL (2025-12-29 04:27:22 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A) context deadline exceeded
======LOGS======
Dec 29 04:26:53.535353088 (564h57m53.406436143s) [upload=495336]: Starting upload
Dec 29 04:26:53.535440746 (564h57m53.406523791s) [upload=495336]: Done starting upload
Dec 29 04:26:53.535532129 (564h57m53.406615164s) [upload=495336]: Uploading segments...
Dec 29 04:26:53.535674981 (564h57m53.406758016s) [upload=495336]: Got next segment. Inline: false
Dec 29 04:26:53.535708846 (564h57m53.406791881s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:26:53.535716401 (564h57m53.406799436s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:26:53.786579547 (564h57m53.657662592s) [seg_pos={0 0}, upload=495336]: Begin upload segment...
Dec 29 04:26:53.786636044 (564h57m53.657719089s) [seg_pos={0 0}, upload=495336]: Done begin upload segment.
Dec 29 04:26:53.786670680 (564h57m53.657753715s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.786742677 (564h57m53.657825722s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787270572 (564h57m53.658353607s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787554293 (564h57m53.658637328s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.787929827 (564h57m53.659012862s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788533297 (564h57m53.659616342s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.788731955 (564h57m53.65981499s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789306048 (564h57m53.660389083s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.789365802 (564h57m53.660448837s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.791670841 (564h57m53.662753876s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792134374 (564h57m53.663217409s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792507183 (564h57m53.663590218s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.792586535 (564h57m53.66366957s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.794347958 (564h57m53.665430993s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.795902809 (564h57m53.666985854s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:53.797609479 (564h57m53.668692524s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Uploading piece...
Dec 29 04:26:55.474759515 (564h57m55.34584255s) [upload=495336]: Next returned nil segment
Dec 29 04:26:55.474777580 (564h57m55.345860615s) [upload=495336]: Waiting for error group managing segments...
Dec 29 04:26:55.760932708 (564h57m55.632015743s) [noise=true , address=145.79.227.18:4006   , piece_id=3BPLWFHSPS42AWWGPIGNB6AU6YGHOJXJUGBOK7SF5TOQ3H7YBF2Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.769087726 (564h57m55.640170771s) [noise=true , address=145.79.227.16:4912   , piece_id=2LEGAANXFLSDPWJ67RU4OPDC6KFXPMFCVSJRBFF67SPMMEA6U2VA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771401953 (564h57m55.642484988s) [noise=true , address=145.79.227.46:4603   , piece_id=JLRCFMEBZDDXPSHMOUTVEWATLPGY5BRXPDEUPAR66P577G2S7BZA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.771490361 (564h57m55.642573396s) [noise=true , address=145.79.227.12:4767   , piece_id=K3GCSTFUU5DOQBZHDJCYKMPMSEWVM25AGA2HW2GN4BCXQ6QQBR6A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.772150528 (564h57m55.643233573s) [noise=true , address=145.79.227.17:4970   , piece_id=FGMDUJ23JYP6WG2XGFSSFMXCNJC47IAVFTMKSSAYPKX543EDVOVA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.778378868 (564h57m55.649461903s) [noise=true , address=145.79.227.48:4301   , piece_id=FPVKUMXHJ2ZCZPAYOTFWF6SRP55OP3M576H62WEAIILG7XI6RUFQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.781420289 (564h57m55.652503324s) [noise=true , address=145.79.227.51:4411   , piece_id=VAC3FF5UX5QCEVDCNEL6N7TBP34JTPMNZGWKONOFT3DG7UVH4TKA, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.783671556 (564h57m55.654754601s) [noise=true , address=145.79.227.41:4085   , piece_id=L7HYCLZXHX6GD754QO7JASHRIXKBPDCZ7LEJBN2MP2MJD5IQJWWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.798090491 (564h57m55.669173526s) [noise=true , address=145.79.227.50:4374   , piece_id=UFUGZKCJ3WKWPOKQ4NDZ22UDOLCHGWRL4VQY3F5SMZQVLSECAVWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.799325433 (564h57m55.670408468s) [noise=true , address=145.79.227.47:4265   , piece_id=23ERUU5GNXZPATSSVLQIEBR22R7VI65ANWIH622U6UM2CUBDKDEQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.800537971 (564h57m55.671621006s) [noise=true , address=145.79.227.3:4151    , piece_id=RRW4ZLAPPWOQOPXCZQSOQV4RFZZWYXNZIXQNVBQXCHWLJLMMPWPQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:55.961790749 (564h57m55.832873784s) [noise=true , address=77.90.31.11:4556     , piece_id=TDZ4VPGLJXOMHHZ3OQ44N3SYCNCXNVFH7PL4KR3UMEWDAGVWOX4A, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014748808 (564h57m55.885831843s) [noise=true , address=45.91.116.7:4009     , piece_id=U5A5Z74PKGCQGH5I65RSSHM46F7IV7MZIUVXT4MTOE3Z5I7ZNSCQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: <nil>
Dec 29 04:26:56.014766792 (564h57m55.885849827s) [seg_pos={0 0}, upload=495336]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:26:56.014933419 (564h57m55.886016454s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.014939531 (564h57m55.886022566s) [noise=true , address=45.91.116.4:4937     , piece_id=CQQ7Y47ELB5PGOH7DGY2CVHYBBT2AWWGAM45CHVGSDBHQC3NCKWQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015202592 (564h57m55.886285627s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015208984 (564h57m55.886292019s) [noise=true , address=145.79.227.52:4447   , piece_id=ZAQJAYB2TNOF4K2IQF7K4XXEVDKH7CIWR5E6IGJUZGKTVYT3JHYQ, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015323402 (564h57m55.886406447s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Dec 29 04:26:56.015329112 (564h57m55.886412157s) [noise=true , address=45.91.116.12:4226    , piece_id=5HZZKVP4CSNIKT73C5KQORL2JSOAQR7O4VL4AO6AZJE5SIEEYP5Q, seg_pos={0 0}, upload=495336]: Long tail cancellation due to optimal threshold
Dec 29 04:26:56.015360833 (564h57m55.886443878s) [seg_pos={0 0}, upload=495336]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:26:56.015423382 (564h57m55.886506427s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:26:56.015429433 (564h57m55.886512468s) [upload=495336]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:26:56.311663423 (564h57m56.182746488s) [upload=495336]: Done uploading segments...
Dec 29 04:26:56.311687018 (564h57m56.182770073s) [upload=495336]: Upload finished. err: <nil>
Duration: 30.002265278s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
13 out of 60 is failed (threshold=6) [X......X.................XX..X.XXXX..X...X.........XX.......]
duration: 
3.538548601s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:27:21 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.538567477s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.X.......X.X......X.............X...........................]
duration: 
27.950330882s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:26:50 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 27.950349929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
14 out of 60 is failed (threshold=6) [XX......X.................XX..X.XXXX..X...X.........XX......]
duration: 
3.554435742s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:26:50 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.554488913s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..X.......X.X......X.............X..........................]
duration: 
41.035341761s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:26:22 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 41.035367831s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [XXX......X.................XX..X.XXXX..X...X.........XX.....]
duration: 
4.679430967s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:26:20 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.679461343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [XXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX.]
duration: 
5.899623875s
trace: 
https://jaeger.dev.storj.io/trace/4616ea0d9c64bcd9
FAIL (2025-12-29 04:26:03 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, Piece ID: PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q) context deadline exceeded
======LOGS======
Dec 29 04:24:54.003222924 (564h55m53.874305969s) [upload=495308]: Starting upload
Dec 29 04:24:54.003309309 (564h55m53.874392354s) [upload=495308]: Done starting upload
Dec 29 04:24:54.003398880 (564h55m53.874481945s) [upload=495308]: Uploading segments...
Dec 29 04:24:54.003508238 (564h55m53.874591283s) [upload=495308]: Got next segment. Inline: false
Dec 29 04:24:54.003572911 (564h55m53.874655976s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:24:54.003579053 (564h55m53.874662108s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:24:54.250774065 (564h55m54.12185715s) [seg_pos={0 0}, upload=495308]: Begin upload segment...
Dec 29 04:24:54.250844339 (564h55m54.121927374s) [seg_pos={0 0}, upload=495308]: Done begin upload segment.
Dec 29 04:24:54.250903492 (564h55m54.121986527s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.250922959 (564h55m54.122006004s) [noise=true , address=145.79.227.15:4875   , piece_id=R4CIVCMKTDBIGNFL53JLSFDGT6FNSQ7ONE4KSDDMTWYA4UJPSUNQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.251689569 (564h55m54.122772604s) [noise=true , address=145.79.227.16:4912   , piece_id=23MTQRYEUVVBN64ZEFETLM3IHY3HHFEHY5KIJ4MFYP34ZSRCJSSA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.252095591 (564h55m54.123178636s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.252542021 (564h55m54.123625057s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253030331 (564h55m54.124113376s) [noise=true , address=145.79.227.46:4602   , piece_id=BJGASXQZ7HD4H7ZMER7LRZ5SRWUENUHZ7262FYSDQDBUUE6SXBLA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253415835 (564h55m54.12449887s) [noise=true , address=45.91.116.5:4298     , piece_id=HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253432867 (564h55m54.124515902s) [noise=true , address=145.79.227.6:4258    , piece_id=U3HEJHKTLRWAIP3UJRZL23FUGJJZ2WOYPCWGX4WS4E475UWY4KKQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253766052 (564h55m54.124849087s) [noise=true , address=145.79.227.26:4654   , piece_id=CNRHWZUKBHRQ3EMQXJ2XRMB4Y6HQTUYN726X7DMR4WWSGUJ2SLBA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254166935 (564h55m54.12524997s) [noise=true , address=145.79.227.36:4519   , piece_id=HY6ZR4SUGJOTS564NDJGXSPJFYE7PWM34YX4CI6NT7GZ5HKHFRVQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254680683 (564h55m54.125763728s) [noise=true , address=145.79.227.14:4840   , piece_id=LIY7ZXMFZA4VNJHOO5DALL7P5XFR5I542PWWJH6L36QXDFCVIO7A, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254899721 (564h55m54.125982756s) [noise=true , address=145.79.227.41:4086   , piece_id=HBY4TAGKNKOD54AZQOFMMSMDW37NMFKFSQAVKWI6XGSJC74GBCEQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.255356691 (564h55m54.126439726s) [noise=true , address=145.79.227.3:4151    , piece_id=2CK27JZCBIQVJZ7VTWR5RGNTPBLWOWW3YVE2YFRYEPNG2QD56O4Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.255701808 (564h55m54.126784844s) [noise=true , address=145.79.227.50:4374   , piece_id=6OXBVEUTG2Z642TS5XBM2N5L6VME3UHMOYD2DCLTUPAULOVSH5DA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.256074157 (564h55m54.127157202s) [noise=true , address=145.79.227.7:4295    , piece_id=P67QUJTD5JSCUUAXTMCXWL4PZJGM2GVOQUQPH5EZT25EY7GJQDPA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.257218075 (564h55m54.12830111s) [noise=true , address=45.91.116.11:4118    , piece_id=PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:55.959684038 (564h55m55.830767073s) [upload=495308]: Next returned nil segment
Dec 29 04:24:55.959697944 (564h55m55.830780979s) [upload=495308]: Waiting for error group managing segments...
Dec 29 04:24:56.239198772 (564h55m56.110281817s) [noise=true , address=145.79.227.6:4258    , piece_id=U3HEJHKTLRWAIP3UJRZL23FUGJJZ2WOYPCWGX4WS4E475UWY4KKQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.243436070 (564h55m56.114519105s) [noise=true , address=145.79.227.41:4086   , piece_id=HBY4TAGKNKOD54AZQOFMMSMDW37NMFKFSQAVKWI6XGSJC74GBCEQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.243488530 (564h55m56.114571575s) [noise=true , address=145.79.227.14:4840   , piece_id=LIY7ZXMFZA4VNJHOO5DALL7P5XFR5I542PWWJH6L36QXDFCVIO7A, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.244957868 (564h55m56.116040903s) [noise=true , address=145.79.227.7:4295    , piece_id=P67QUJTD5JSCUUAXTMCXWL4PZJGM2GVOQUQPH5EZT25EY7GJQDPA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.249083824 (564h55m56.120166859s) [noise=true , address=145.79.227.15:4875   , piece_id=R4CIVCMKTDBIGNFL53JLSFDGT6FNSQ7ONE4KSDDMTWYA4UJPSUNQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.258707399 (564h55m56.129790434s) [noise=true , address=145.79.227.46:4602   , piece_id=BJGASXQZ7HD4H7ZMER7LRZ5SRWUENUHZ7262FYSDQDBUUE6SXBLA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.259598425 (564h55m56.13068146s) [noise=true , address=145.79.227.26:4654   , piece_id=CNRHWZUKBHRQ3EMQXJ2XRMB4Y6HQTUYN726X7DMR4WWSGUJ2SLBA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.261653468 (564h55m56.132736503s) [noise=true , address=145.79.227.3:4151    , piece_id=2CK27JZCBIQVJZ7VTWR5RGNTPBLWOWW3YVE2YFRYEPNG2QD56O4Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.270305042 (564h55m56.141388077s) [noise=true , address=145.79.227.16:4912   , piece_id=23MTQRYEUVVBN64ZEFETLM3IHY3HHFEHY5KIJ4MFYP34ZSRCJSSA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.270446100 (564h55m56.141529146s) [noise=true , address=145.79.227.50:4374   , piece_id=6OXBVEUTG2Z642TS5XBM2N5L6VME3UHMOYD2DCLTUPAULOVSH5DA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.503519774 (564h55m56.374602809s) [noise=true , address=145.79.227.36:4519   , piece_id=HY6ZR4SUGJOTS564NDJGXSPJFYE7PWM34YX4CI6NT7GZ5HKHFRVQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.543216792 (564h55m56.414299827s) [noise=true , address=45.91.116.5:4298     , piece_id=HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.610355203 (564h55m56.481438238s) [noise=true , address=45.91.116.11:4118    , piece_id=PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.610375973 (564h55m56.481459018s) [seg_pos={0 0}, upload=495308]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:24:56.610628934 (564h55m56.481711969s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610638482 (564h55m56.481721527s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.610846939 (564h55m56.481929974s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610876846 (564h55m56.481959911s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.610983309 (564h55m56.482066344s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610993608 (564h55m56.482076643s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.611019277 (564h55m56.482102312s) [seg_pos={0 0}, upload=495308]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:24:56.611126762 (564h55m56.482209797s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:24:56.611132352 (564h55m56.482215387s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:24:56.906804622 (564h55m56.777887668s) [upload=495308]: Done uploading segments...
Dec 29 04:24:56.906823658 (564h55m56.777906693s) [upload=495308]: Upload finished. err: <nil>
Duration: 5.899640897s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [...X.......X.X......X.............X.........................]
duration: 
37.54383703s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:25:34 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 37.54385295s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
36 out of 60 is failed (threshold=3) [XXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..XX]
duration: 
30.004293669s
trace: 
https://jaeger.dev.storj.io/trace/4616ea0d9c64bcd9
FAIL (2025-12-29 04:25:23 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, Piece ID: PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q) context deadline exceeded
======LOGS======
Dec 29 04:24:54.003222924 (564h55m53.874305969s) [upload=495308]: Starting upload
Dec 29 04:24:54.003309309 (564h55m53.874392354s) [upload=495308]: Done starting upload
Dec 29 04:24:54.003398880 (564h55m53.874481945s) [upload=495308]: Uploading segments...
Dec 29 04:24:54.003508238 (564h55m53.874591283s) [upload=495308]: Got next segment. Inline: false
Dec 29 04:24:54.003572911 (564h55m53.874655976s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:24:54.003579053 (564h55m53.874662108s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:24:54.250774065 (564h55m54.12185715s) [seg_pos={0 0}, upload=495308]: Begin upload segment...
Dec 29 04:24:54.250844339 (564h55m54.121927374s) [seg_pos={0 0}, upload=495308]: Done begin upload segment.
Dec 29 04:24:54.250903492 (564h55m54.121986527s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.250922959 (564h55m54.122006004s) [noise=true , address=145.79.227.15:4875   , piece_id=R4CIVCMKTDBIGNFL53JLSFDGT6FNSQ7ONE4KSDDMTWYA4UJPSUNQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.251689569 (564h55m54.122772604s) [noise=true , address=145.79.227.16:4912   , piece_id=23MTQRYEUVVBN64ZEFETLM3IHY3HHFEHY5KIJ4MFYP34ZSRCJSSA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.252095591 (564h55m54.123178636s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.252542021 (564h55m54.123625057s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253030331 (564h55m54.124113376s) [noise=true , address=145.79.227.46:4602   , piece_id=BJGASXQZ7HD4H7ZMER7LRZ5SRWUENUHZ7262FYSDQDBUUE6SXBLA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253415835 (564h55m54.12449887s) [noise=true , address=45.91.116.5:4298     , piece_id=HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253432867 (564h55m54.124515902s) [noise=true , address=145.79.227.6:4258    , piece_id=U3HEJHKTLRWAIP3UJRZL23FUGJJZ2WOYPCWGX4WS4E475UWY4KKQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.253766052 (564h55m54.124849087s) [noise=true , address=145.79.227.26:4654   , piece_id=CNRHWZUKBHRQ3EMQXJ2XRMB4Y6HQTUYN726X7DMR4WWSGUJ2SLBA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254166935 (564h55m54.12524997s) [noise=true , address=145.79.227.36:4519   , piece_id=HY6ZR4SUGJOTS564NDJGXSPJFYE7PWM34YX4CI6NT7GZ5HKHFRVQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254680683 (564h55m54.125763728s) [noise=true , address=145.79.227.14:4840   , piece_id=LIY7ZXMFZA4VNJHOO5DALL7P5XFR5I542PWWJH6L36QXDFCVIO7A, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.254899721 (564h55m54.125982756s) [noise=true , address=145.79.227.41:4086   , piece_id=HBY4TAGKNKOD54AZQOFMMSMDW37NMFKFSQAVKWI6XGSJC74GBCEQ, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.255356691 (564h55m54.126439726s) [noise=true , address=145.79.227.3:4151    , piece_id=2CK27JZCBIQVJZ7VTWR5RGNTPBLWOWW3YVE2YFRYEPNG2QD56O4Q, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.255701808 (564h55m54.126784844s) [noise=true , address=145.79.227.50:4374   , piece_id=6OXBVEUTG2Z642TS5XBM2N5L6VME3UHMOYD2DCLTUPAULOVSH5DA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.256074157 (564h55m54.127157202s) [noise=true , address=145.79.227.7:4295    , piece_id=P67QUJTD5JSCUUAXTMCXWL4PZJGM2GVOQUQPH5EZT25EY7GJQDPA, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:54.257218075 (564h55m54.12830111s) [noise=true , address=45.91.116.11:4118    , piece_id=PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A, seg_pos={0 0}, upload=495308]: Uploading piece...
Dec 29 04:24:55.959684038 (564h55m55.830767073s) [upload=495308]: Next returned nil segment
Dec 29 04:24:55.959697944 (564h55m55.830780979s) [upload=495308]: Waiting for error group managing segments...
Dec 29 04:24:56.239198772 (564h55m56.110281817s) [noise=true , address=145.79.227.6:4258    , piece_id=U3HEJHKTLRWAIP3UJRZL23FUGJJZ2WOYPCWGX4WS4E475UWY4KKQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.243436070 (564h55m56.114519105s) [noise=true , address=145.79.227.41:4086   , piece_id=HBY4TAGKNKOD54AZQOFMMSMDW37NMFKFSQAVKWI6XGSJC74GBCEQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.243488530 (564h55m56.114571575s) [noise=true , address=145.79.227.14:4840   , piece_id=LIY7ZXMFZA4VNJHOO5DALL7P5XFR5I542PWWJH6L36QXDFCVIO7A, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.244957868 (564h55m56.116040903s) [noise=true , address=145.79.227.7:4295    , piece_id=P67QUJTD5JSCUUAXTMCXWL4PZJGM2GVOQUQPH5EZT25EY7GJQDPA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.249083824 (564h55m56.120166859s) [noise=true , address=145.79.227.15:4875   , piece_id=R4CIVCMKTDBIGNFL53JLSFDGT6FNSQ7ONE4KSDDMTWYA4UJPSUNQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.258707399 (564h55m56.129790434s) [noise=true , address=145.79.227.46:4602   , piece_id=BJGASXQZ7HD4H7ZMER7LRZ5SRWUENUHZ7262FYSDQDBUUE6SXBLA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.259598425 (564h55m56.13068146s) [noise=true , address=145.79.227.26:4654   , piece_id=CNRHWZUKBHRQ3EMQXJ2XRMB4Y6HQTUYN726X7DMR4WWSGUJ2SLBA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.261653468 (564h55m56.132736503s) [noise=true , address=145.79.227.3:4151    , piece_id=2CK27JZCBIQVJZ7VTWR5RGNTPBLWOWW3YVE2YFRYEPNG2QD56O4Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.270305042 (564h55m56.141388077s) [noise=true , address=145.79.227.16:4912   , piece_id=23MTQRYEUVVBN64ZEFETLM3IHY3HHFEHY5KIJ4MFYP34ZSRCJSSA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.270446100 (564h55m56.141529146s) [noise=true , address=145.79.227.50:4374   , piece_id=6OXBVEUTG2Z642TS5XBM2N5L6VME3UHMOYD2DCLTUPAULOVSH5DA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.503519774 (564h55m56.374602809s) [noise=true , address=145.79.227.36:4519   , piece_id=HY6ZR4SUGJOTS564NDJGXSPJFYE7PWM34YX4CI6NT7GZ5HKHFRVQ, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.543216792 (564h55m56.414299827s) [noise=true , address=45.91.116.5:4298     , piece_id=HSVF5YJ7EASNQR6HGEJJMYMU3LEETHMQBCBSUM4PTATVDLMC7J7Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.610355203 (564h55m56.481438238s) [noise=true , address=45.91.116.11:4118    , piece_id=PFRSFBG5WEUW7OLN65DFYQO5Z6EJQKKKFHDCHXCGUUMN2AVDLP3A, seg_pos={0 0}, upload=495308]: Done uploading piece. err: <nil>
Dec 29 04:24:56.610375973 (564h55m56.481459018s) [seg_pos={0 0}, upload=495308]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:24:56.610628934 (564h55m56.481711969s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610638482 (564h55m56.481721527s) [noise=true , address=145.79.227.40:4051   , piece_id=VDTEW3YZ2HUFWPVT3KQRGLBCL6KYC5N7AKJBAVQ5HOSRMO4LHCVA, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.610846939 (564h55m56.481929974s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610876846 (564h55m56.481959911s) [noise=true , address=45.91.116.8:4045     , piece_id=6SCY4BW6HQBK67XYH3MB52KQDJ7VAE4JPOTARECGDSIUWHKC3QTA, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.610983309 (564h55m56.482066344s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:56.610993608 (564h55m56.482076643s) [noise=true , address=77.90.31.17:4770     , piece_id=ZG6QREOAR52DT6RYZ3ZZETETKH4RJY6F2CEE4TMTSQEPDIYZQW6Q, seg_pos={0 0}, upload=495308]: Long tail cancellation due to optimal threshold
Dec 29 04:24:56.611019277 (564h55m56.482102312s) [seg_pos={0 0}, upload=495308]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:24:56.611126762 (564h55m56.482209797s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:24:56.611132352 (564h55m56.482215387s) [upload=495308]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:24:56.906804622 (564h55m56.777887668s) [upload=495308]: Done uploading segments...
Dec 29 04:24:56.906823658 (564h55m56.777906693s) [upload=495308]: Upload finished. err: <nil>
Duration: 30.004315001s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [.XXX......X.................XX..X.XXXX..X...X.........XX....]
duration: 
5.001179544s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:25:21 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001200444s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [....X.......X.X......X.............X........................]
duration: 
37.357203257s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:24:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 37.357234406s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [..XXX......X.................XX..X.XXXX..X...X.........XX...]
duration: 
5.001122505s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:24:54 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001152121s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
36 out of 60 is failed (threshold=3) [XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..X]
duration: 
30.002637604s
trace: 
https://jaeger.dev.storj.io/trace/1b50b02509b489a1
FAIL (2025-12-29 04:24:46 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9 04:24:17.188067619 (564h55m17.059150664s) [upload=495294]: Starting upload
Dec 29 04:24:17.188142903 (564h55m17.059225948s) [upload=495294]: Done starting upload
Dec 29 04:24:17.188244256 (564h55m17.059327301s) [upload=495294]: Uploading segments...
Dec 29 04:24:17.188594482 (564h55m17.059677518s) [upload=495294]: Got next segment. Inline: false
Dec 29 04:24:17.188636633 (564h55m17.059719678s) [upload=495294]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:24:17.188641812 (564h55m17.059724847s) [upload=495294]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:24:17.434669202 (564h55m17.305752247s) [seg_pos={0 0}, upload=495294]: Begin upload segment...
Dec 29 04:24:17.434745127 (564h55m17.305828162s) [seg_pos={0 0}, upload=495294]: Done begin upload segment.
Dec 29 04:24:17.434813176 (564h55m17.305896221s) [noise=true , address=145.79.227.18:4006   , piece_id=TZNSAKR3M5K5ZLSYJZEHDIUWVATQA267K3GCEURPCPHIOEVLJ5ZA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.434868962 (564h55m17.305952007s) [noise=true , address=145.79.227.2:4113    , piece_id=RZ5YIGZTJ4RNY7SA2EQ3DCVGIYJHRTMRGOJRG43Y3XJA6EVDTUMQ, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.435472081 (564h55m17.306555116s) [noise=true , address=77.90.31.14:4663     , piece_id=BKETZA6QZ3CQIIEUSZVULBN52YK7YKABZ7YMR3KXANADYMWY2BTA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.435882513 (564h55m17.306965558s) [noise=true , address=145.79.227.11:4729   , piece_id=PH7HXWP5DZ4LOY522SQ6VQM5BAJIUANF4MYX24IZH5JOK4JK5ONA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.436208293 (564h55m17.307291338s) [noise=true , address=145.79.227.17:4970   , piece_id=2FXT2R23IPVEP6MUJVJCPWNRU7TMJHZU3SVPQAVSJN7NE3NARFWA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.436564691 (564h55m17.307647736s) [noise=true , address=145.79.227.12:4767   , piece_id=LB7HSIP7YFU6CSXM7JOEURXIXW43L6XGVQG6NQRBX3ZK6ATJBVPQ, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.437920363 (564h55m17.309003398s) [noise=true , address=145.79.227.52:4445   , piece_id=FJUQLI6CMCXILISB5GVSQ3URJWJ45MAUFIFDO4BLZYU2PWBJM44Q, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.438646435 (564h55m17.30972948s) [noise=true , address=45.91.116.9:4263     , piece_id=WU3CUIJOLOEIBWDZQUBMFN5PLVMU2RMRMPHFDKF4VETPRBOGHO6Q, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.439827264 (564h55m17.310910299s) [noise=true , address=145.79.227.26:4653   , piece_id=LW5BS4YNTH5GF5Y5D3LALFWWNDP2ASSPNCXZCN4VGCHUJLQOZ26A, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.440322266 (564h55m17.311405311s) [noise=true , address=77.90.31.13:4629     , piece_id=ZHWAGC6RUU37AEONPUWIGAFU46XPZJBGLVGZ2GFOTF4WJTVRRFAA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.441503826 (564h55m17.312586861s) [noise=true , address=145.79.227.14:4838   , piece_id=VT356WJG3TJOWXLBKINQRI2G3W5FK62JE2YY2XBBC45MORVRC4OA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.443176060 (564h55m17.314259106s) [noise=true , address=145.79.227.37:4566   , piece_id=FMCBCCCUU4DQCCWHTHIGHLJGEXQMB3LUW7VQQ7JZPLTLTTISXYOQ, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.444656369 (564h55m17.315739404s) [noise=true , address=77.90.31.10:4518     , piece_id=HJNRMBI5PYO6KD3KSNVF5CP3DW44NZ7PXRI2WE23OS64RW6ESMWA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.446267526 (564h55m17.317350561s) [noise=true , address=145.79.227.20:4438   , piece_id=IKWULOCG4EXUPDTIFYXSH6USHSOF7RHPXS3UFMVTSOQMH57QT64Q, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.448035223 (564h55m17.319118258s) [noise=true , address=45.91.116.8:4045     , piece_id=DM6NX5T4PJJWZPSI7JSAHCRYY22XGIGHSFGCJOX27F6Z3CBCGQXA, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:17.449692098 (564h55m17.320775133s) [noise=true , address=145.79.227.3:4151    , piece_id=DOJX5MQAXVZ66WAE5FGRNWNSTA6ZHJBEVGATPDLX6ECSPGXR2G4Q, seg_pos={0 0}, upload=495294]: Uploading piece...
Dec 29 04:24:19.135053531 (564h55m19.006136566s) [upload=495294]: Next returned nil segment
Dec 29 04:24:19.135065414 (564h55m19.006148449s) [upload=495294]: Waiting for error group managing segments...
Dec 29 04:24:19.414188340 (564h55m19.285271376s) [noise=true , address=145.79.227.18:4006   , piece_id=TZNSAKR3M5K5ZLSYJZEHDIUWVATQA267K3GCEURPCPHIOEVLJ5ZA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.415443921 (564h55m19.286526956s) [noise=true , address=145.79.227.17:4970   , piece_id=2FXT2R23IPVEP6MUJVJCPWNRU7TMJHZU3SVPQAVSJN7NE3NARFWA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.416727996 (564h55m19.287811042s) [noise=true , address=145.79.227.12:4767   , piece_id=LB7HSIP7YFU6CSXM7JOEURXIXW43L6XGVQG6NQRBX3ZK6ATJBVPQ, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.428958826 (564h55m19.300041871s) [noise=true , address=145.79.227.52:4445   , piece_id=FJUQLI6CMCXILISB5GVSQ3URJWJ45MAUFIFDO4BLZYU2PWBJM44Q, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.429327789 (564h55m19.300410824s) [noise=true , address=145.79.227.14:4838   , piece_id=VT356WJG3TJOWXLBKINQRI2G3W5FK62JE2YY2XBBC45MORVRC4OA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.429431376 (564h55m19.300514411s) [noise=true , address=145.79.227.26:4653   , piece_id=LW5BS4YNTH5GF5Y5D3LALFWWNDP2ASSPNCXZCN4VGCHUJLQOZ26A, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.433713750 (564h55m19.304796795s) [noise=true , address=145.79.227.3:4151    , piece_id=DOJX5MQAXVZ66WAE5FGRNWNSTA6ZHJBEVGATPDLX6ECSPGXR2G4Q, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.437041937 (564h55m19.308124982s) [noise=true , address=145.79.227.11:4729   , piece_id=PH7HXWP5DZ4LOY522SQ6VQM5BAJIUANF4MYX24IZH5JOK4JK5ONA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.447713527 (564h55m19.318796572s) [noise=true , address=145.79.227.2:4113    , piece_id=RZ5YIGZTJ4RNY7SA2EQ3DCVGIYJHRTMRGOJRG43Y3XJA6EVDTUMQ, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.451233930 (564h55m19.322316976s) [noise=true , address=145.79.227.20:4438   , piece_id=IKWULOCG4EXUPDTIFYXSH6USHSOF7RHPXS3UFMVTSOQMH57QT64Q, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.465509674 (564h55m19.336592719s) [noise=true , address=145.79.227.37:4566   , piece_id=FMCBCCCUU4DQCCWHTHIGHLJGEXQMB3LUW7VQQ7JZPLTLTTISXYOQ, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.668828150 (564h55m19.539911185s) [noise=true , address=77.90.31.13:4629     , piece_id=ZHWAGC6RUU37AEONPUWIGAFU46XPZJBGLVGZ2GFOTF4WJTVRRFAA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.721312166 (564h55m19.592395211s) [noise=true , address=77.90.31.14:4663     , piece_id=BKETZA6QZ3CQIIEUSZVULBN52YK7YKABZ7YMR3KXANADYMWY2BTA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: <nil>
Dec 29 04:24:19.721339067 (564h55m19.592422112s) [seg_pos={0 0}, upload=495294]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:24:19.721662182 (564h55m19.592745227s) [noise=true , address=45.91.116.9:4263     , piece_id=WU3CUIJOLOEIBWDZQUBMFN5PLVMU2RMRMPHFDKF4VETPRBOGHO6Q, seg_pos={0 0}, upload=495294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: write tcp 188.166.225.170:41678->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:41678->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41678->45.91.116.9:4263: use of closed network connection
Dec 29 04:24:19.721672773 (564h55m19.592755818s) [noise=true , address=45.91.116.9:4263     , piece_id=WU3CUIJOLOEIBWDZQUBMFN5PLVMU2RMRMPHFDKF4VETPRBOGHO6Q, seg_pos={0 0}, upload=495294]: Long tail cancellation due to optimal threshold
Dec 29 04:24:19.721677612 (564h55m19.592760667s) [noise=true , address=77.90.31.10:4518     , piece_id=HJNRMBI5PYO6KD3KSNVF5CP3DW44NZ7PXRI2WE23OS64RW6ESMWA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:19.721688823 (564h55m19.592771868s) [noise=true , address=77.90.31.10:4518     , piece_id=HJNRMBI5PYO6KD3KSNVF5CP3DW44NZ7PXRI2WE23OS64RW6ESMWA, seg_pos={0 0}, upload=495294]: Long tail cancellation due to optimal threshold
Dec 29 04:24:19.721997832 (564h55m19.593080887s) [noise=true , address=45.91.116.8:4045     , piece_id=DM6NX5T4PJJWZPSI7JSAHCRYY22XGIGHSFGCJOX27F6Z3CBCGQXA, seg_pos={0 0}, upload=495294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 29 04:24:19.722021196 (564h55m19.593104241s) [noise=true , address=45.91.116.8:4045     , piece_id=DM6NX5T4PJJWZPSI7JSAHCRYY22XGIGHSFGCJOX27F6Z3CBCGQXA, seg_pos={0 0}, upload=495294]: Long tail cancellation due to optimal threshold
Dec 29 04:24:19.722044179 (564h55m19.593127225s) [seg_pos={0 0}, upload=495294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:24:19.722137337 (564h55m19.593220382s) [upload=495294]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:24:19.722144781 (564h55m19.593227826s) [upload=495294]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:24:20.013609249 (564h55m19.884692294s) [upload=495294]: Done uploading segments...
Dec 29 04:24:20.013636179 (564h55m19.884719225s) [upload=495294]: Upload finished. err: <nil>
Duration: 30.002660106s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [...XXX......X.................XX..X.XXXX..X...X.........XX..]
duration: 
3.546994212s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:24:01 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.547014883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.....X.......X.X......X.............X.......................]
duration: 
15.509701901s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:23:49 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 15.50971694s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [.XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X..]
duration: 
6.110591309s
trace: 
https://jaeger.dev.storj.io/trace/4732bdf59e82e0b4
FAIL (2025-12-29 04:23:48 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q) context deadline exceeded; piecestore: (Node ID: 129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H, Piece ID: 6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ) context deadline exceeded
======LOGS======
Dec 29 04:22:24.017045089 (564h53m23.888128124s) [upload=495268]: Starting upload
Dec 29 04:22:24.017099833 (564h53m23.888182878s) [upload=495268]: Done starting upload
Dec 29 04:22:24.017275678 (564h53m23.888358713s) [upload=495268]: Uploading segments...
Dec 29 04:22:24.017456542 (564h53m23.888539577s) [upload=495268]: Got next segment. Inline: false
Dec 29 04:22:24.017508471 (564h53m23.888591516s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:22:24.017515153 (564h53m23.888598198s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:22:24.263501214 (564h53m24.134584249s) [seg_pos={0 0}, upload=495268]: Begin upload segment...
Dec 29 04:22:24.263568191 (564h53m24.134651227s) [seg_pos={0 0}, upload=495268]: Done begin upload segment.
Dec 29 04:22:24.263648003 (564h53m24.134731058s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263664425 (564h53m24.13474748s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263759686 (564h53m24.134842721s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264205044 (564h53m24.135288079s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264621517 (564h53m24.135704552s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265009786 (564h53m24.136092821s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265416481 (564h53m24.136499526s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265794050 (564h53m24.136877095s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266331803 (564h53m24.137414849s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266351481 (564h53m24.137434526s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266759187 (564h53m24.137842222s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266925233 (564h53m24.138008268s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.267460263 (564h53m24.138543298s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268235328 (564h53m24.139318363s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268569093 (564h53m24.139652138s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268880707 (564h53m24.139963742s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:25.861187195 (564h53m25.73227023s) [upload=495268]: Next returned nil segment
Dec 29 04:22:25.861200812 (564h53m25.732283847s) [upload=495268]: Waiting for error group managing segments...
Dec 29 04:22:26.241481543 (564h53m26.112564588s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.247957164 (564h53m26.119040209s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.254467360 (564h53m26.125550405s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.261254364 (564h53m26.132337409s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.264007055 (564h53m26.13509009s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.275454453 (564h53m26.146537498s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.281878926 (564h53m26.152961961s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.289112530 (564h53m26.160195575s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.442217839 (564h53m26.313300874s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.520757613 (564h53m26.391840658s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.538893988 (564h53m26.409977023s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.565498307 (564h53m26.436581342s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539267783 (564h53m27.410350818s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539292380 (564h53m27.410375435s) [seg_pos={0 0}, upload=495268]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:22:27.539521226 (564h53m27.410604271s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539530002 (564h53m27.410613038s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539607279 (564h53m27.410690324s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539618481 (564h53m27.410701516s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539785639 (564h53m27.410868694s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539800658 (564h53m27.410883703s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539823691 (564h53m27.410906736s) [seg_pos={0 0}, upload=495268]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:22:27.539918442 (564h53m27.411001487s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:22:27.539924874 (564h53m27.411007919s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:22:27.824381058 (564h53m27.695464103s) [upload=495268]: Done uploading segments...
Dec 29 04:22:27.824398622 (564h53m27.695481667s) [upload=495268]: Upload finished. err: <nil>
Duration: 6.110621308s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [......X.......X.X......X.............X......................]
duration: 
51.146997747s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:23:33 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 51.147012896s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [....XXX......X.................XX..X.XXXX..X...X.........XX.]
duration: 
3.511544231s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:23:31 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.511560631s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [..XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X.]
duration: 
6.074234841s
trace: 
https://jaeger.dev.storj.io/trace/4732bdf59e82e0b4
FAIL (2025-12-29 04:22:59 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q) context deadline exceeded; piecestore: (Node ID: 129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H, Piece ID: 6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ) context deadline exceeded
======LOGS======
Dec 29 04:22:24.017045089 (564h53m23.888128124s) [upload=495268]: Starting upload
Dec 29 04:22:24.017099833 (564h53m23.888182878s) [upload=495268]: Done starting upload
Dec 29 04:22:24.017275678 (564h53m23.888358713s) [upload=495268]: Uploading segments...
Dec 29 04:22:24.017456542 (564h53m23.888539577s) [upload=495268]: Got next segment. Inline: false
Dec 29 04:22:24.017508471 (564h53m23.888591516s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:22:24.017515153 (564h53m23.888598198s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:22:24.263501214 (564h53m24.134584249s) [seg_pos={0 0}, upload=495268]: Begin upload segment...
Dec 29 04:22:24.263568191 (564h53m24.134651227s) [seg_pos={0 0}, upload=495268]: Done begin upload segment.
Dec 29 04:22:24.263648003 (564h53m24.134731058s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263664425 (564h53m24.13474748s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263759686 (564h53m24.134842721s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264205044 (564h53m24.135288079s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264621517 (564h53m24.135704552s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265009786 (564h53m24.136092821s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265416481 (564h53m24.136499526s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265794050 (564h53m24.136877095s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266331803 (564h53m24.137414849s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266351481 (564h53m24.137434526s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266759187 (564h53m24.137842222s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266925233 (564h53m24.138008268s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.267460263 (564h53m24.138543298s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268235328 (564h53m24.139318363s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268569093 (564h53m24.139652138s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268880707 (564h53m24.139963742s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:25.861187195 (564h53m25.73227023s) [upload=495268]: Next returned nil segment
Dec 29 04:22:25.861200812 (564h53m25.732283847s) [upload=495268]: Waiting for error group managing segments...
Dec 29 04:22:26.241481543 (564h53m26.112564588s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.247957164 (564h53m26.119040209s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.254467360 (564h53m26.125550405s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.261254364 (564h53m26.132337409s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.264007055 (564h53m26.13509009s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.275454453 (564h53m26.146537498s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.281878926 (564h53m26.152961961s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.289112530 (564h53m26.160195575s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.442217839 (564h53m26.313300874s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.520757613 (564h53m26.391840658s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.538893988 (564h53m26.409977023s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.565498307 (564h53m26.436581342s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539267783 (564h53m27.410350818s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539292380 (564h53m27.410375435s) [seg_pos={0 0}, upload=495268]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:22:27.539521226 (564h53m27.410604271s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539530002 (564h53m27.410613038s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539607279 (564h53m27.410690324s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539618481 (564h53m27.410701516s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539785639 (564h53m27.410868694s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539800658 (564h53m27.410883703s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539823691 (564h53m27.410906736s) [seg_pos={0 0}, upload=495268]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:22:27.539918442 (564h53m27.411001487s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:22:27.539924874 (564h53m27.411007919s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:22:27.824381058 (564h53m27.695464103s) [upload=495268]: Done uploading segments...
Dec 29 04:22:27.824398622 (564h53m27.695481667s) [upload=495268]: Upload finished. err: <nil>
Duration: 6.07425026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [...XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.X]
duration: 
30.003351598s
trace: 
https://jaeger.dev.storj.io/trace/4732bdf59e82e0b4
FAIL (2025-12-29 04:22:53 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A) context deadline exceeded; piecestore: (Node ID: 125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, Piece ID: O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q) context deadline exceeded; piecestore: (Node ID: 129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H, Piece ID: 6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ) context deadline exceeded
======LOGS======
Dec 29 04:22:24.017045089 (564h53m23.888128124s) [upload=495268]: Starting upload
Dec 29 04:22:24.017099833 (564h53m23.888182878s) [upload=495268]: Done starting upload
Dec 29 04:22:24.017275678 (564h53m23.888358713s) [upload=495268]: Uploading segments...
Dec 29 04:22:24.017456542 (564h53m23.888539577s) [upload=495268]: Got next segment. Inline: false
Dec 29 04:22:24.017508471 (564h53m23.888591516s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:22:24.017515153 (564h53m23.888598198s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:22:24.263501214 (564h53m24.134584249s) [seg_pos={0 0}, upload=495268]: Begin upload segment...
Dec 29 04:22:24.263568191 (564h53m24.134651227s) [seg_pos={0 0}, upload=495268]: Done begin upload segment.
Dec 29 04:22:24.263648003 (564h53m24.134731058s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263664425 (564h53m24.13474748s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.263759686 (564h53m24.134842721s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264205044 (564h53m24.135288079s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.264621517 (564h53m24.135704552s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265009786 (564h53m24.136092821s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265416481 (564h53m24.136499526s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.265794050 (564h53m24.136877095s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266331803 (564h53m24.137414849s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266351481 (564h53m24.137434526s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266759187 (564h53m24.137842222s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.266925233 (564h53m24.138008268s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.267460263 (564h53m24.138543298s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268235328 (564h53m24.139318363s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268569093 (564h53m24.139652138s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:24.268880707 (564h53m24.139963742s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Uploading piece...
Dec 29 04:22:25.861187195 (564h53m25.73227023s) [upload=495268]: Next returned nil segment
Dec 29 04:22:25.861200812 (564h53m25.732283847s) [upload=495268]: Waiting for error group managing segments...
Dec 29 04:22:26.241481543 (564h53m26.112564588s) [noise=true , address=145.79.227.13:4803   , piece_id=K3CHEHHKIX2BKKULTZE2IPR2Q7AZCYWRBMLTGLOK3SOQIZ4BEN2Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.247957164 (564h53m26.119040209s) [noise=true , address=145.79.227.2:4113    , piece_id=CNYV3Z5VZC4XTUYQKUHBEWOJFGZLCWNDW3L2X3XET5MUVAJMN2HA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.254467360 (564h53m26.125550405s) [noise=true , address=145.79.227.19:4079   , piece_id=QOJ66RWHW7HQBQKTUXFHZ3JZ2IU34XK3PVA2DI2EBQYDQCFYVXQA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.261254364 (564h53m26.132337409s) [noise=true , address=145.79.227.15:4876   , piece_id=5UNFKLS42QA7OFOUVNIYY7MIPJQMJGMVZECM7MNI5B6CX3WTXWKQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.264007055 (564h53m26.13509009s) [noise=true , address=145.79.227.32:4835   , piece_id=S4KRFZU32D4INIOHO36DKF5D4JEAFAFQSYYDCZ77QFXDMBBYXYQQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.275454453 (564h53m26.146537498s) [noise=true , address=145.79.227.48:4302   , piece_id=DKRJLLR7L5GEQHFJQMN65BNXGJLKMVEYFOBIK7SDA5RGDFKQSUPA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.281878926 (564h53m26.152961961s) [noise=true , address=145.79.227.52:4447   , piece_id=F25C3WNVGSY7QSCVKRU3GZQJVGGQJLDLTRA3KHMRPOHJLEVVJOIQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.289112530 (564h53m26.160195575s) [noise=true , address=145.79.227.47:4265   , piece_id=TVATQEQQZ4NODP7TBLQIWD4OA2YXXCFPRT3BB6WK3FIBOLFQ6ELA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.442217839 (564h53m26.313300874s) [noise=true , address=77.90.31.11:4555     , piece_id=4M3O5YVPTMUUFO5IKS3FV3SQK3PGCQFYGRQ2V2UP4E4MY5YFMLBA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.520757613 (564h53m26.391840658s) [noise=true , address=45.91.116.8:4046     , piece_id=6QKO56MZT7XCMRUSAK5DLVBMU3XXZ7756EJEZK65XE5ZDBEH2AYQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.538893988 (564h53m26.409977023s) [noise=true , address=45.91.116.7:4010     , piece_id=H7KFNEMYPGVIESOYXGVZKP6A3J7DPSOHR2VTQKMFAYQ6UAHHDW3A, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:26.565498307 (564h53m26.436581342s) [noise=true , address=77.90.31.14:4661     , piece_id=5MNX6FGVLONVBF73MRGBR46GE2GYMXA7CPQF2MXBQNBHSBMA6RFA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539267783 (564h53m27.410350818s) [noise=true , address=77.90.31.17:4770     , piece_id=O3ABUJWNWK223JV4ZE46LYPIBVINNIG32MR7NTMRWDELBYXJL67Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: <nil>
Dec 29 04:22:27.539292380 (564h53m27.410375435s) [seg_pos={0 0}, upload=495268]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:22:27.539521226 (564h53m27.410604271s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539530002 (564h53m27.410613038s) [noise=true , address=145.79.227.30:4761   , piece_id=ANHQIMDWRWMRUTBPOCYGNXOMYKTEG2LK2WDFT3K4OJYVOQST27RA, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539607279 (564h53m27.410690324s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539618481 (564h53m27.410701516s) [noise=true , address=145.79.227.43:4157   , piece_id=AVYNMGLCSUYI24YSEPGC5YCGX55D6THJCSOWILQ4TOJ4B5DFLLVQ, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539785639 (564h53m27.410868694s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF
Dec 29 04:22:27.539800658 (564h53m27.410883703s) [noise=true , address=45.91.116.9:4261     , piece_id=HFIBJDN2YM7K6GA347YKQ5BHNXWPO3NNOBKRCB66Z4E6UHDOVV5Q, seg_pos={0 0}, upload=495268]: Long tail cancellation due to optimal threshold
Dec 29 04:22:27.539823691 (564h53m27.410906736s) [seg_pos={0 0}, upload=495268]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:22:27.539918442 (564h53m27.411001487s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:22:27.539924874 (564h53m27.411007919s) [upload=495268]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:22:27.824381058 (564h53m27.695464103s) [upload=495268]: Done uploading segments...
Dec 29 04:22:27.824398622 (564h53m27.695481667s) [upload=495268]: Upload finished. err: <nil>
Duration: 30.003386954s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.......X.......X.X......X.............X.....................]
duration: 
35.747816098s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:22:41 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 35.747867695s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [.....XXX......X.................XX..X.XXXX..X...X.........XX]
duration: 
5.000959626s
trace: 
https://jaeger.dev.storj.io/trace/2cc24f49d4662732
FAIL (2025-12-29 04:22:28 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000983864s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
35 out of 60 is failed (threshold=3) [X...XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X.]
duration: 
6.06551673s
trace: 
https://jaeger.dev.storj.io/trace/7b1971a2434d1e53
FAIL (2025-12-29 04:21:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA) context deadline exceeded; piecestore: (Node ID: 12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, Piece ID: DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ) context deadline exceeded
======LOGS======
gments...
Dec 29 04:20:32.033735922 (564h51m31.904818967s) [upload=495241]: Got next segment. Inline: false
Dec 29 04:20:32.033778542 (564h51m31.904861578s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:20:32.033784755 (564h51m31.90486779s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:20:32.278392330 (564h51m32.149475375s) [seg_pos={0 0}, upload=495241]: Begin upload segment...
Dec 29 04:20:32.278460099 (564h51m32.149543145s) [seg_pos={0 0}, upload=495241]: Done begin upload segment.
Dec 29 04:20:32.278492942 (564h51m32.149575987s) [noise=true , address=145.79.227.14:4837   , piece_id=6XWTCLQI7UTUNGGCAQK5U4ZJI45B3KQDWK34L6ZBXZ66LREWNDMQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.279489840 (564h51m32.150572885s) [noise=true , address=145.79.227.46:4602   , piece_id=3NR4SLVJZPVQW3CVIWQ7W2WURWVFHUUTY3ANBM24XFXWJXHUT3RA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.279909709 (564h51m32.150992754s) [noise=true , address=77.90.31.10:4518     , piece_id=PWVZYKPQRFNL4B5M2AZNRLEC4FNYVXMYOIXMHZDUNZMQOWW53DGQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280196666 (564h51m32.151279711s) [noise=true , address=45.91.116.7:4009     , piece_id=CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280530702 (564h51m32.151613737s) [noise=true , address=145.79.227.48:4302   , piece_id=QNMNR7D5JLIVOFCS6PAISIXGMTQZ7RIY26RKG752JICUU52FIOJQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280863366 (564h51m32.151946411s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281187983 (564h51m32.152271049s) [noise=true , address=77.90.31.12:4589     , piece_id=UWEN4EMMDZDGZEX2SWK2LHGCCWDO6CTOSQGBDNXAKHVYK7F2HBAQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281499557 (564h51m32.152582592s) [noise=true , address=145.79.227.20:4438   , piece_id=2Q472DNDTAIGD7N3XRHQF24ZOSJHBOWF54QW7E63ZFPT6N7KEPGA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281847730 (564h51m32.152930775s) [noise=true , address=45.91.116.6:4973     , piece_id=DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282194210 (564h51m32.153277245s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282557090 (564h51m32.153640135s) [noise=true , address=145.79.227.12:4765   , piece_id=3W33F2OF5FJV5T5RNVQK4J2G3OFC662ST6WEQASXP7L4ILSZIOOA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282859566 (564h51m32.153942601s) [noise=true , address=145.79.227.19:4077   , piece_id=GSMSAY2CS3IYGUWFJPKS44UHC7F4CXBWRXWOKHMOGMCGBYK6W62Q, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283156292 (564h51m32.154239327s) [noise=true , address=145.79.227.47:4267   , piece_id=5I7CBFLH5QXBMILSHBKJCTJYCJIWFTQ4CGVQY2XUNQDXJ6RFRBQQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283466773 (564h51m32.154549818s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283793866 (564h51m32.154876901s) [noise=true , address=145.79.227.7:4294    , piece_id=BOSUUWESTAMDMQ5KAGNEX46XWOTEKF2AU4M75G6EZOGGGAAHU3DQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.284237992 (564h51m32.155321027s) [noise=true , address=145.79.227.17:4970   , piece_id=XPWJCO4IVTYJA4JXCPPAOQTQ63ZGANH3ABAU2E67UAYCPQGRFBFQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:33.982448830 (564h51m33.853531875s) [upload=495241]: Next returned nil segment
Dec 29 04:20:33.982463778 (564h51m33.853546823s) [upload=495241]: Waiting for error group managing segments...
Dec 29 04:20:34.265707754 (564h51m34.136790789s) [noise=true , address=145.79.227.14:4837   , piece_id=6XWTCLQI7UTUNGGCAQK5U4ZJI45B3KQDWK34L6ZBXZ66LREWNDMQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.267942449 (564h51m34.139025484s) [noise=true , address=145.79.227.17:4970   , piece_id=XPWJCO4IVTYJA4JXCPPAOQTQ63ZGANH3ABAU2E67UAYCPQGRFBFQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.268285272 (564h51m34.139368307s) [noise=true , address=145.79.227.12:4765   , piece_id=3W33F2OF5FJV5T5RNVQK4J2G3OFC662ST6WEQASXP7L4ILSZIOOA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.270379079 (564h51m34.141462114s) [noise=true , address=145.79.227.19:4077   , piece_id=GSMSAY2CS3IYGUWFJPKS44UHC7F4CXBWRXWOKHMOGMCGBYK6W62Q, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.270697346 (564h51m34.141780391s) [noise=true , address=145.79.227.7:4294    , piece_id=BOSUUWESTAMDMQ5KAGNEX46XWOTEKF2AU4M75G6EZOGGGAAHU3DQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.272354531 (564h51m34.143437566s) [noise=true , address=145.79.227.20:4438   , piece_id=2Q472DNDTAIGD7N3XRHQF24ZOSJHBOWF54QW7E63ZFPT6N7KEPGA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.274282903 (564h51m34.145365938s) [noise=true , address=145.79.227.48:4302   , piece_id=QNMNR7D5JLIVOFCS6PAISIXGMTQZ7RIY26RKG752JICUU52FIOJQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.287186052 (564h51m34.158269087s) [noise=true , address=145.79.227.46:4602   , piece_id=3NR4SLVJZPVQW3CVIWQ7W2WURWVFHUUTY3ANBM24XFXWJXHUT3RA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.304337402 (564h51m34.175420437s) [noise=true , address=145.79.227.47:4267   , piece_id=5I7CBFLH5QXBMILSHBKJCTJYCJIWFTQ4CGVQY2XUNQDXJ6RFRBQQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.470798783 (564h51m34.341881818s) [noise=true , address=77.90.31.10:4518     , piece_id=PWVZYKPQRFNL4B5M2AZNRLEC4FNYVXMYOIXMHZDUNZMQOWW53DGQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.503626583 (564h51m34.374709618s) [noise=true , address=77.90.31.12:4589     , piece_id=UWEN4EMMDZDGZEX2SWK2LHGCCWDO6CTOSQGBDNXAKHVYK7F2HBAQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.543948133 (564h51m34.415031178s) [noise=true , address=45.91.116.7:4009     , piece_id=CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.649530329 (564h51m34.520613364s) [noise=true , address=45.91.116.6:4973     , piece_id=DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.649550247 (564h51m34.520633282s) [seg_pos={0 0}, upload=495241]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:20:34.649733136 (564h51m34.520816171s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection
Dec 29 04:20:34.649738586 (564h51m34.520821621s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.649878703 (564h51m34.520961738s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection; write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection
Dec 29 04:20:34.649886527 (564h51m34.520969572s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.650035491 (564h51m34.521118526s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz): protocol: expected piece hash; context canceled; EOF
Dec 29 04:20:34.650041593 (564h51m34.521124638s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.650095916 (564h51m34.521178951s) [seg_pos={0 0}, upload=495241]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:20:34.650171069 (564h51m34.521254104s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:20:34.650175768 (564h51m34.521258813s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:20:34.941646820 (564h51m34.812729855s) [upload=495241]: Done uploading segments...
Dec 29 04:20:34.941665215 (564h51m34.812748251s) [upload=495241]: Upload finished. err: <nil>
Duration: 6.065541145s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
14 out of 60 is failed (threshold=6) [......XXX......X.................XX..X.XXXX..X...X.........X]
duration: 
5.001510534s
trace: 
https://jaeger.dev.storj.io/trace/6ab9a2d4efcfecf6
FAIL (2025-12-29 04:21:27 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001537135s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [........X.......X.X......X.............X....................]
duration: 
15.412915239s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:21:08 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 15.412942601s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
36 out of 60 is failed (threshold=3) [XX...XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...X]
duration: 
30.003393607s
trace: 
https://jaeger.dev.storj.io/trace/7b1971a2434d1e53
FAIL (2025-12-29 04:21:01 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA) context deadline exceeded; piecestore: (Node ID: 12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, Piece ID: DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ) context deadline exceeded
======LOGS======
gments...
Dec 29 04:20:32.033735922 (564h51m31.904818967s) [upload=495241]: Got next segment. Inline: false
Dec 29 04:20:32.033778542 (564h51m31.904861578s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:20:32.033784755 (564h51m31.90486779s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:20:32.278392330 (564h51m32.149475375s) [seg_pos={0 0}, upload=495241]: Begin upload segment...
Dec 29 04:20:32.278460099 (564h51m32.149543145s) [seg_pos={0 0}, upload=495241]: Done begin upload segment.
Dec 29 04:20:32.278492942 (564h51m32.149575987s) [noise=true , address=145.79.227.14:4837   , piece_id=6XWTCLQI7UTUNGGCAQK5U4ZJI45B3KQDWK34L6ZBXZ66LREWNDMQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.279489840 (564h51m32.150572885s) [noise=true , address=145.79.227.46:4602   , piece_id=3NR4SLVJZPVQW3CVIWQ7W2WURWVFHUUTY3ANBM24XFXWJXHUT3RA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.279909709 (564h51m32.150992754s) [noise=true , address=77.90.31.10:4518     , piece_id=PWVZYKPQRFNL4B5M2AZNRLEC4FNYVXMYOIXMHZDUNZMQOWW53DGQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280196666 (564h51m32.151279711s) [noise=true , address=45.91.116.7:4009     , piece_id=CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280530702 (564h51m32.151613737s) [noise=true , address=145.79.227.48:4302   , piece_id=QNMNR7D5JLIVOFCS6PAISIXGMTQZ7RIY26RKG752JICUU52FIOJQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.280863366 (564h51m32.151946411s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281187983 (564h51m32.152271049s) [noise=true , address=77.90.31.12:4589     , piece_id=UWEN4EMMDZDGZEX2SWK2LHGCCWDO6CTOSQGBDNXAKHVYK7F2HBAQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281499557 (564h51m32.152582592s) [noise=true , address=145.79.227.20:4438   , piece_id=2Q472DNDTAIGD7N3XRHQF24ZOSJHBOWF54QW7E63ZFPT6N7KEPGA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.281847730 (564h51m32.152930775s) [noise=true , address=45.91.116.6:4973     , piece_id=DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282194210 (564h51m32.153277245s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282557090 (564h51m32.153640135s) [noise=true , address=145.79.227.12:4765   , piece_id=3W33F2OF5FJV5T5RNVQK4J2G3OFC662ST6WEQASXP7L4ILSZIOOA, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.282859566 (564h51m32.153942601s) [noise=true , address=145.79.227.19:4077   , piece_id=GSMSAY2CS3IYGUWFJPKS44UHC7F4CXBWRXWOKHMOGMCGBYK6W62Q, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283156292 (564h51m32.154239327s) [noise=true , address=145.79.227.47:4267   , piece_id=5I7CBFLH5QXBMILSHBKJCTJYCJIWFTQ4CGVQY2XUNQDXJ6RFRBQQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283466773 (564h51m32.154549818s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.283793866 (564h51m32.154876901s) [noise=true , address=145.79.227.7:4294    , piece_id=BOSUUWESTAMDMQ5KAGNEX46XWOTEKF2AU4M75G6EZOGGGAAHU3DQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:32.284237992 (564h51m32.155321027s) [noise=true , address=145.79.227.17:4970   , piece_id=XPWJCO4IVTYJA4JXCPPAOQTQ63ZGANH3ABAU2E67UAYCPQGRFBFQ, seg_pos={0 0}, upload=495241]: Uploading piece...
Dec 29 04:20:33.982448830 (564h51m33.853531875s) [upload=495241]: Next returned nil segment
Dec 29 04:20:33.982463778 (564h51m33.853546823s) [upload=495241]: Waiting for error group managing segments...
Dec 29 04:20:34.265707754 (564h51m34.136790789s) [noise=true , address=145.79.227.14:4837   , piece_id=6XWTCLQI7UTUNGGCAQK5U4ZJI45B3KQDWK34L6ZBXZ66LREWNDMQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.267942449 (564h51m34.139025484s) [noise=true , address=145.79.227.17:4970   , piece_id=XPWJCO4IVTYJA4JXCPPAOQTQ63ZGANH3ABAU2E67UAYCPQGRFBFQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.268285272 (564h51m34.139368307s) [noise=true , address=145.79.227.12:4765   , piece_id=3W33F2OF5FJV5T5RNVQK4J2G3OFC662ST6WEQASXP7L4ILSZIOOA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.270379079 (564h51m34.141462114s) [noise=true , address=145.79.227.19:4077   , piece_id=GSMSAY2CS3IYGUWFJPKS44UHC7F4CXBWRXWOKHMOGMCGBYK6W62Q, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.270697346 (564h51m34.141780391s) [noise=true , address=145.79.227.7:4294    , piece_id=BOSUUWESTAMDMQ5KAGNEX46XWOTEKF2AU4M75G6EZOGGGAAHU3DQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.272354531 (564h51m34.143437566s) [noise=true , address=145.79.227.20:4438   , piece_id=2Q472DNDTAIGD7N3XRHQF24ZOSJHBOWF54QW7E63ZFPT6N7KEPGA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.274282903 (564h51m34.145365938s) [noise=true , address=145.79.227.48:4302   , piece_id=QNMNR7D5JLIVOFCS6PAISIXGMTQZ7RIY26RKG752JICUU52FIOJQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.287186052 (564h51m34.158269087s) [noise=true , address=145.79.227.46:4602   , piece_id=3NR4SLVJZPVQW3CVIWQ7W2WURWVFHUUTY3ANBM24XFXWJXHUT3RA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.304337402 (564h51m34.175420437s) [noise=true , address=145.79.227.47:4267   , piece_id=5I7CBFLH5QXBMILSHBKJCTJYCJIWFTQ4CGVQY2XUNQDXJ6RFRBQQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.470798783 (564h51m34.341881818s) [noise=true , address=77.90.31.10:4518     , piece_id=PWVZYKPQRFNL4B5M2AZNRLEC4FNYVXMYOIXMHZDUNZMQOWW53DGQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.503626583 (564h51m34.374709618s) [noise=true , address=77.90.31.12:4589     , piece_id=UWEN4EMMDZDGZEX2SWK2LHGCCWDO6CTOSQGBDNXAKHVYK7F2HBAQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.543948133 (564h51m34.415031178s) [noise=true , address=45.91.116.7:4009     , piece_id=CWMX7CXB5E7WX2TPZW7FMOROHHPVXRFJEKM6T72UZ3TWWHNJUQRA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.649530329 (564h51m34.520613364s) [noise=true , address=45.91.116.6:4973     , piece_id=DDH3APTRUOP65B3XD64WW3I2EPAD5KK75YGA3YGLS7JLVFZYHGCQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: <nil>
Dec 29 04:20:34.649550247 (564h51m34.520633282s) [seg_pos={0 0}, upload=495241]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:20:34.649733136 (564h51m34.520816171s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52600->77.90.31.16:4735: use of closed network connection
Dec 29 04:20:34.649738586 (564h51m34.520821621s) [noise=true , address=77.90.31.16:4735     , piece_id=GTQ3T4TKGWJ5B4EBIC2DG2UE65YN6T7U6GZY3JD5XDZHABWII7GA, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.649878703 (564h51m34.520961738s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection; write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40354->77.90.31.18:4808: use of closed network connection
Dec 29 04:20:34.649886527 (564h51m34.520969572s) [noise=true , address=77.90.31.18:4808     , piece_id=JIWZFW7J7YHAEHRNJINEVBSHWBW3DFSMGJFDUR3ABALKWNCVRCRQ, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.650035491 (564h51m34.521118526s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz): protocol: expected piece hash; context canceled; EOF
Dec 29 04:20:34.650041593 (564h51m34.521124638s) [noise=true , address=145.79.227.29:4943   , piece_id=BL3QS7TQ23J4UGRGEOLQ7OELDCMJ6BWSMKQJW5YDHXX5PF6IZG6Q, seg_pos={0 0}, upload=495241]: Long tail cancellation due to optimal threshold
Dec 29 04:20:34.650095916 (564h51m34.521178951s) [seg_pos={0 0}, upload=495241]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:20:34.650171069 (564h51m34.521254104s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:20:34.650175768 (564h51m34.521258813s) [upload=495241]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:20:34.941646820 (564h51m34.812729855s) [upload=495241]: Done uploading segments...
Dec 29 04:20:34.941665215 (564h51m34.812748251s) [upload=495241]: Upload finished. err: <nil>
Duration: 30.003418986s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.........X.......X.X......X.............X...................]
duration: 
15.560713814s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:20:40 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 15.560737759s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
14 out of 60 is failed (threshold=6) [X......XXX......X.................XX..X.XXXX..X...X.........]
duration: 
3.469624368s
trace: 
https://jaeger.dev.storj.io/trace/3992892a3b2f0cca
FAIL (2025-12-29 04:20:40 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.46970425s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
14 out of 60 is failed (threshold=6) [.X......XXX......X.................XX..X.XXXX..X...X........]
duration: 
4.225230163s
trace: 
https://jaeger.dev.storj.io/trace/3992892a3b2f0cca
FAIL (2025-12-29 04:20:24 UTC) 
orbiter warning: context deadline exceeded
Duration: 4.225461263s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..........X.......X.X......X.............X..................]
duration: 
35.88064777s
trace: 
https://jaeger.dev.storj.io/trace/72beea211c54dceb
FAIL (2025-12-29 04:20:16 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
 04:05:05.301177067 (564h36m5.172260112s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301468672 (564h36m5.172551717s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301878142 (564h36m5.172961187s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.301892169 (564h36m5.172975214s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302216927 (564h36m5.173299972s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302327137 (564h36m5.173410182s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302684628 (564h36m5.173767673s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.302932379 (564h36m5.174015424s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303033903 (564h36m5.174116948s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303385993 (564h36m5.174469038s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303457709 (564h36m5.174540754s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303825861 (564h36m5.174908906s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.303866187 (564h36m5.174949233s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304215443 (564h36m5.175298488s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304302198 (564h36m5.175385243s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:05.304739651 (564h36m5.175822696s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Uploading piece...
Dec 29 04:05:06.612294683 (564h36m6.483377728s) [upload=495011]: Next returned nil segment
Dec 29 04:05:06.612312939 (564h36m6.483395984s) [upload=495011]: Waiting for error group managing segments...
Dec 29 04:05:07.137646212 (564h36m7.008729257s) [noise=true , address=145.79.227.6:4266    , piece_id=VARLMNUO5V4T5KFYD6BQ3XHENCVVPQNRVYTWNV3NCGVHGTRHK75Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.153061636 (564h36m7.024144681s) [noise=true , address=145.79.227.46:4625   , piece_id=LHJG3OGMBKV7FIYCT2PHXRSGE2PBS2ZLBGMEPLCMHVXDXRPHQ4AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.174839534 (564h36m7.045922579s) [noise=true , address=145.79.227.2:4130    , piece_id=QVIKJQ7OOZJZYXWZ5UCJFQGXRJKXAWJ34IZ6PFKP46FLIBYI5UTQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.196190780 (564h36m7.067273825s) [noise=true , address=145.79.227.13:4831   , piece_id=AYEVGOIXOHA3U3ZUJB4DRLBXZRAQPHPM554QYOIF5CEKXT7SQ6HA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.439229803 (564h36m7.310312839s) [noise=true , address=45.91.116.9:4274     , piece_id=3WC4AZ6K3W45F6AYRPL7GXBEWMOAAVIQLVRKNBFK7ENMENZEPPJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.458345231 (564h36m7.329428266s) [noise=true , address=45.91.116.5:4309     , piece_id=SPLHWRKDB5XBYTMHA35AT3PW4MNVDPUJVTTKUGEY7Z5E4SSLAOJA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.461742912 (564h36m7.332825957s) [noise=true , address=77.90.31.17:4799     , piece_id=CKKO5SEZCKBQRLNFZVJKMLGCVDAFTPOICUSMVCDM7URTGFFKNYKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.501492300 (564h36m7.372575335s) [noise=true , address=45.91.116.10:4107    , piece_id=HQWXGMVKAU7LH5XBBJDM2VN3YBEZHFNYWH4NRRKBYGC7WYGBV6AQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.520950161 (564h36m7.392033196s) [noise=true , address=45.91.116.14:4204    , piece_id=6ZXLIAXBHNISDBIUAJV73SHVSFVBRI4N2BND67C6AVFUBJJHE45Q, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:07.900960668 (564h36m7.772043713s) [noise=true , address=193.149.164.143:4699 , piece_id=E7LTOGDKLAZHYNTU4KX4HBAYYT3MACHAAVH736JLJXRNN3OOJVKQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.070127046 (564h36m7.941210081s) [noise=true , address=77.90.31.15:4705     , piece_id=GJPXFW3E7UNYOOBQLG3TKZCWI6U7ORSAVFAU4MZ5RZBTN75LI6IQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.116327979 (564h36m7.987411015s) [noise=true , address=145.79.227.44:4201   , piece_id=H2QP7ULE7GIA5SDKJXEX5S7C5SVWP2474UP36JF4PXMPKTAMZVKA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.117842052 (564h36m7.988925097s) [noise=true , address=145.79.227.40:4081   , piece_id=WQKZITIAU7COGCM3XF7MM5JB64WB4GFMQKPUCBHNUY6IWJVWX4TA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145319343 (564h36m8.016402378s) [noise=true , address=45.91.116.8:4065     , piece_id=QGDKNF4XZYRPYLWZHUCP2IRJFR3E4A7ABSU7RUOK4NCEUBF2KKEQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: <nil>
Dec 29 04:05:08.145349340 (564h36m8.016432386s) [seg_pos={0 0}, upload=495011]: Segment reached optimal threshold of 14 pieces.
Dec 29 04:05:08.145520887 (564h36m8.016603932s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKJiP9crUtUHuDmbJhbfbuvZDDpfsXJ17N7LJUE3KDMUGMCkk): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.145527079 (564h36m8.016610124s) [noise=true , address=145.79.227.30:4796   , piece_id=23D3V4P6NBGQMJBO7UWWUWD7ABA2URIZIMYRYRPTHOBQWQAHAGUA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.145961806 (564h36m8.017044851s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54220->64.49.13.169:4706: use of closed network connection
Dec 29 04:05:08.145971706 (564h36m8.017054741s) [noise=true , address=64.49.13.169:4706    , piece_id=U7FALR7T6ZBLQWSZY55EMCTCVUQVOCE3FXTZKUF43MMUEGIGEBFQ, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146249024 (564h36m8.017332069s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146259193 (564h36m8.017342248s) [noise=true , address=145.79.227.33:4873   , piece_id=43LPXE4YYOYDHIJMJNZM4JZV56EXQVUDYIOQD37RMXFGBAWTN5IA, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146601144 (564h36m8.017684189s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vVHmSRAfugvyvTkEZHitE4uFMGXxgrbnWfJuGD9MzS9avAYsU): protocol: expected piece hash; context canceled; EOF
Dec 29 04:05:08.146612105 (564h36m8.01769515s) [noise=true , address=145.79.227.29:4947   , piece_id=5DJOP4RIA76SP32KTRL5GC6HWJS6YGQNYPBEWFXHYELLIYBA7U3A, seg_pos={0 0}, upload=495011]: Long tail cancellation due to optimal threshold
Dec 29 04:05:08.146660708 (564h36m8.017743753s) [seg_pos={0 0}, upload=495011]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 04:05:08.146791666 (564h36m8.017874712s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:05:08.146799432 (564h36m8.017882477s) [upload=495011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:05:08.432716311 (564h36m8.303799356s) [upload=495011]: Done uploading segments...
Dec 29 04:05:08.432736209 (564h36m8.303819254s) [upload=495011]: Upload finished. err: <nil>
Duration: 35.88066362s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
36 out of 60 is failed (threshold=3) [XXX...XXXXXXX.XX.XXX.XXXX.XXX.X..X..XXX.X..X.XX.X.X.X.X.X...]
duration: 
5.674575847s
trace: 
https://jaeger.dev.storj.io/trace/178afbae1e3822fb
FAIL (2025-12-29 04:20:04 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, Piece ID: YBDV76PPPDGXGKU4IARW4DXW7GIU3WMN7NLXNLRLQ4RNNEWTB7UA) context deadline exceeded; piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: OOU5Z3FXJLLGJGHUTJLQMY4UJYTBQ2JDZ4AWHECADEKGDHP5TIBQ) context deadline exceeded
======LOGS======
Dec 29 04:17:33.306043407 (564h48m33.177126452s) [upload=495188]: Starting upload
Dec 29 04:17:33.306136304 (564h48m33.177219349s) [upload=495188]: Done starting upload
Dec 29 04:17:33.306238098 (564h48m33.177321153s) [upload=495188]: Uploading segments...
Dec 29 04:17:33.306436426 (564h48m33.177519471s) [upload=495188]: Got next segment. Inline: false
Dec 29 04:17:33.306476522 (564h48m33.177559567s) [upload=495188]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 04:17:33.306484136 (564h48m33.177567181s) [upload=495188]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 04:17:33.554079780 (564h48m33.425162816s) [seg_pos={0 0}, upload=495188]: Begin upload segment...
Dec 29 04:17:33.554131189 (564h48m33.425214224s) [seg_pos={0 0}, upload=495188]: Done begin upload segment.
Dec 29 04:17:33.554153391 (564h48m33.425236426s) [noise=true , address=145.79.227.6:4257    , piece_id=RPBAPFBQH7GDYFVFMGVDCP6Y7W6TRTLPTE2DLOVXX372P6667SBA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.554160384 (564h48m33.42524342s) [noise=true , address=145.79.227.37:4567   , piece_id=CS6TRJIJPIVUOSDVTODTX7YOHSSZAOK2KR7V3VZUZ4OMDGN2HY5Q, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.554670635 (564h48m33.42575367s) [noise=true , address=145.79.227.17:4971   , piece_id=7NDPAJA747ZBP44OTAQCQ2BQLCC5BAUAJLZII6ZX5EL4D3EVG53A, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.555028999 (564h48m33.426112034s) [noise=true , address=45.91.116.14:4190    , piece_id=ON46ARLJBZEKOPXDMO5Z4SEREK7TVSIJSZEB33JCHTYY6VOASL4A, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.555302790 (564h48m33.426385835s) [noise=true , address=145.79.227.49:4339   , piece_id=3V3TN5R74UFYPKKA635LV2OSIEMOAY4FUW6U5NP4BVERJJWUG6HA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.555711187 (564h48m33.426794222s) [noise=true , address=145.79.227.41:4085   , piece_id=YPHCST7MO5FTCGD56KS7OWUNIJC3GRUEEERUUC226PPHWXFY37UA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.556245866 (564h48m33.427328911s) [noise=true , address=45.91.116.12:4227    , piece_id=RVXM72ID6B5XLB34CKRBPFQRQ5HG5U5HQS2IXCYGDRFR4R7FXI4Q, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.556569482 (564h48m33.427652527s) [noise=true , address=145.79.227.42:4122   , piece_id=I5SCDN3FMYQ5YSQJJ2RATVRP6OKVVPXJ3SDGF7I6OKJ35Y27HYRQ, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.556944546 (564h48m33.428027591s) [noise=true , address=45.91.116.11:4117    , piece_id=UT2ZCC7WQPMSGQ6DTXCRLTC2FZ7PGXLFYOBVJJRDAPLHV7VOLUUA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.558219092 (564h48m33.429302127s) [noise=true , address=45.91.116.7:4009     , piece_id=J4HF6VMJZBOB3WUOYH5DB4V3O73J3LU3E2QB6KYXJXGXONWUJUJQ, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.559737123 (564h48m33.430820158s) [noise=true , address=45.91.116.8:4045     , piece_id=YBDV76PPPDGXGKU4IARW4DXW7GIU3WMN7NLXNLRLQ4RNNEWTB7UA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.561414487 (564h48m33.432497522s) [noise=true , address=145.79.227.13:4802   , piece_id=WFUZRWLHGCC2BFAPPDKVISFKMVZ6QAJQC3XSFT2IAFOLD6YLDR4A, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.562715696 (564h48m33.433798741s) [noise=true , address=77.90.31.21:4878     , piece_id=OOU5Z3FXJLLGJGHUTJLQMY4UJYTBQ2JDZ4AWHECADEKGDHP5TIBQ, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.563860775 (564h48m33.43494381s) [noise=true , address=145.79.227.12:4765   , piece_id=LYHD2C7AJCCAXK7A5CEUUBX42TDL6VPDDJKUMHOQKMTMD2QYZ4AA, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.564849058 (564h48m33.435932093s) [noise=true , address=145.79.227.15:4873   , piece_id=AK3BO7WSVAIHXJHP24EXGA5PTCLVGWAMNWXEBD7Y27ABICUEOFMQ, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:33.565923884 (564h48m33.437006929s) [noise=true , address=145.79.227.51:4410   , piece_id=UWSSPU2MOVKRUWCYIGUOPCTJLOIYJFLH4NPMFKZNPLDYKTRXNO3A, seg_pos={0 0}, upload=495188]: Uploading piece...
Dec 29 04:17:35.250915200 (564h48m35.121998245s) [upload=495188]: Next returned nil segment
Dec 29 04:17:35.250928746 (564h48m35.122011781s) [upload=495188]: Waiting for error group managing segments...
Dec 29 04:17:35.536282108 (564h48m35.407365154s) [noise=true , address=145.79.227.6:4257    , piece_id=RPBAPFBQH7GDYFVFMGVDCP6Y7W6TRTLPTE2DLOVXX372P6667SBA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.537178746 (564h48m35.408261791s) [noise=true , address=145.79.227.17:4971   , piece_id=7NDPAJA747ZBP44OTAQCQ2BQLCC5BAUAJLZII6ZX5EL4D3EVG53A, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.540461296 (564h48m35.411544331s) [noise=true , address=145.79.227.13:4802   , piece_id=WFUZRWLHGCC2BFAPPDKVISFKMVZ6QAJQC3XSFT2IAFOLD6YLDR4A, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.542728934 (564h48m35.413811969s) [noise=true , address=145.79.227.12:4765   , piece_id=LYHD2C7AJCCAXK7A5CEUUBX42TDL6VPDDJKUMHOQKMTMD2QYZ4AA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.543185504 (564h48m35.414268549s) [noise=true , address=145.79.227.42:4122   , piece_id=I5SCDN3FMYQ5YSQJJ2RATVRP6OKVVPXJ3SDGF7I6OKJ35Y27HYRQ, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.545900394 (564h48m35.416983439s) [noise=true , address=145.79.227.15:4873   , piece_id=AK3BO7WSVAIHXJHP24EXGA5PTCLVGWAMNWXEBD7Y27ABICUEOFMQ, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.558478234 (564h48m35.429561269s) [noise=true , address=145.79.227.37:4567   , piece_id=CS6TRJIJPIVUOSDVTODTX7YOHSSZAOK2KR7V3VZUZ4OMDGN2HY5Q, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.563292761 (564h48m35.434375807s) [noise=true , address=145.79.227.41:4085   , piece_id=YPHCST7MO5FTCGD56KS7OWUNIJC3GRUEEERUUC226PPHWXFY37UA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.575589046 (564h48m35.446672091s) [noise=true , address=145.79.227.51:4410   , piece_id=UWSSPU2MOVKRUWCYIGUOPCTJLOIYJFLH4NPMFKZNPLDYKTRXNO3A, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:35.810597342 (564h48m35.681680377s) [noise=true , address=45.91.116.8:4045     , piece_id=YBDV76PPPDGXGKU4IARW4DXW7GIU3WMN7NLXNLRLQ4RNNEWTB7UA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:36.095308282 (564h48m35.966391327s) [noise=true , address=45.91.116.12:4227    , piece_id=RVXM72ID6B5XLB34CKRBPFQRQ5HG5U5HQS2IXCYGDRFR4R7FXI4Q, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:36.107625867 (564h48m35.978708912s) [noise=true , address=45.91.116.7:4009     , piece_id=J4HF6VMJZBOB3WUOYH5DB4V3O73J3LU3E2QB6KYXJXGXONWUJUJQ, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:36.475530611 (564h48m36.346613656s) [noise=true , address=77.90.31.21:4878     , piece_id=OOU5Z3FXJLLGJGHUTJLQMY4UJYTBQ2JDZ4AWHECADEKGDHP5TIBQ, seg_pos={0 0}, upload=495188]: Done uploading piece. err: <nil>
Dec 29 04:17:36.475552824 (564h48m36.346635859s) [seg_pos={0 0}, upload=495188]: Segment reached optimal threshold of 13 pieces.
Dec 29 04:17:36.475743677 (564h48m36.346826712s) [noise=true , address=45.91.116.11:4117    , piece_id=UT2ZCC7WQPMSGQ6DTXCRLTC2FZ7PGXLFYOBVJJRDAPLHV7VOLUUA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF
Dec 29 04:17:36.475750279 (564h48m36.346833324s) [noise=true , address=45.91.116.11:4117    , piece_id=UT2ZCC7WQPMSGQ6DTXCRLTC2FZ7PGXLFYOBVJJRDAPLHV7VOLUUA, seg_pos={0 0}, upload=495188]: Long tail cancellation due to optimal threshold
Dec 29 04:17:36.475832687 (564h48m36.346915722s) [noise=true , address=145.79.227.49:4339   , piece_id=3V3TN5R74UFYPKKA635LV2OSIEMOAY4FUW6U5NP4BVERJJWUG6HA, seg_pos={0 0}, upload=495188]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Dec 29 04:17:36.475844759 (564h48m36.346927804s) [noise=true , address=145.79.227.49:4339   , piece_id=3V3TN5R74UFYPKKA635LV2OSIEMOAY4FUW6U5NP4BVERJJWUG6HA, seg_pos={0 0}, upload=495188]: Long tail cancellation due to optimal threshold
Dec 29 04:17:36.476083454 (564h48m36.347166489s) [noise=true , address=45.91.116.14:4190    , piece_id=ON46ARLJBZEKOPXDMO5Z4SEREK7TVSIJSZEB33JCHTYY6VOASL4A, seg_pos={0 0}, upload=495188]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Dec 29 04:17:36.476091840 (564h48m36.347174895s) [noise=true , address=45.91.116.14:4190    , piece_id=ON46ARLJBZEKOPXDMO5Z4SEREK7TVSIJSZEB33JCHTYY6VOASL4A, seg_pos={0 0}, upload=495188]: Long tail cancellation due to optimal threshold
Dec 29 04:17:36.476106909 (564h48m36.347189944s) [seg_pos={0 0}, upload=495188]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 04:17:36.476179798 (564h48m36.347262833s) [upload=495188]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 04:17:36.476185789 (564h48m36.347268824s) [upload=495188]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 04:17:36.764714998 (564h48m36.635798033s) [upload=495188]: Done uploading segments...
Dec 29 04:17:36.764735998 (564h48m36.635819043s) [upload=495188]: Upload finished. err: <nil>
Duration: 5.674602319s core uplink us-select-1