Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 906586 reports (Page 11255 of 18132)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [........................................X...................]
duration: 
38.015122464s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:04:20 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 38.015139425s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [...XXX..X.X.X.......XX....X..X.XX.........X..X......X......X]
duration: 
5.001306221s
trace: 
https://jaeger.dev.storj.io/trace/4e970421565a5f56
FAIL (2025-12-30 10:04:08 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001333402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [X...X...X..............X....X.X..X.X..............XX.X.XX.X.]
duration: 
5.715918296s
trace: 
https://jaeger.dev.storj.io/trace/5f0586ec7b21b1c6
FAIL (2025-12-30 10:03:54 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, Piece ID: 5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ) context deadline exceeded
======LOGS======
30 10:03:17.905926101 (594h34m17.777009146s) [upload=521214]: Starting upload
Dec 30 10:03:17.905986765 (594h34m17.7770698s) [upload=521214]: Done starting upload
Dec 30 10:03:17.906714621 (594h34m17.777797657s) [upload=521214]: Uploading segments...
Dec 30 10:03:17.906764287 (594h34m17.777847342s) [upload=521214]: Got next segment. Inline: false
Dec 30 10:03:17.906822046 (594h34m17.777905091s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 10:03:17.906828439 (594h34m17.777911474s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 10:03:18.153761891 (594h34m18.024844926s) [seg_pos={0 0}, upload=521214]: Begin upload segment...
Dec 30 10:03:18.153854618 (594h34m18.024937653s) [seg_pos={0 0}, upload=521214]: Done begin upload segment.
Dec 30 10:03:18.153886619 (594h34m18.024969664s) [noise=true , address=77.90.31.15:4700     , piece_id=D6U4KZFVXQGZBIWJ5ZXCJPVZEJ2WNVUAN4MXBSNBGHOGXVJA6LLA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154059038 (594h34m18.025142083s) [noise=true , address=45.91.116.8:4045     , piece_id=VIABQZSLCZ4ZYNMUH2S2YOR35DDAQUPYEGVHJLZJMMDI7CE5QZNA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154741607 (594h34m18.025824652s) [noise=true , address=77.90.31.10:4518     , piece_id=YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154912632 (594h34m18.025995667s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155074782 (594h34m18.026157817s) [noise=true , address=145.79.227.39:4013   , piece_id=2JEJAAA3RALU6ZGHNFH3YIUR5TPF3OQOO73DRGNDZ5T4FU3NKTKA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155294379 (594h34m18.026377424s) [noise=true , address=145.79.227.51:4410   , piece_id=UDEQMSHOZM2G46N3SCMFPSPJLBT5OISWARG7PLY4UETPC7FK3GBQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155467429 (594h34m18.026550464s) [noise=true , address=77.90.31.16:4736     , piece_id=X4SSVDEJPFV47ZJ4TPIFMJ5B4MOFWGPJWEQM5NSOJYTL7PNLSTLQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155563773 (594h34m18.026646808s) [noise=true , address=45.91.116.11:4119    , piece_id=5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155782479 (594h34m18.026865514s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155890825 (594h34m18.02697387s) [noise=true , address=145.79.227.27:4691   , piece_id=HB426ZFOAWVW7BTNBVJVMZVWLMAP45ASCCCKTR7GQ6N57LYU34VA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156115724 (594h34m18.027198769s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156234289 (594h34m18.027317334s) [noise=true , address=77.90.31.11:4555     , piece_id=N2GUODEX2INTREDGKMRLLJ7RX35F2L7YWBEBUPKGPSU3ZFR44RLA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156488924 (594h34m18.027571959s) [noise=true , address=45.91.116.7:4010     , piece_id=XGNLR5TU5QE2L3H5KRYNY3E42JYWJZMLCQ7D2CURNJMLDAOV4GCQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156514503 (594h34m18.027597538s) [noise=true , address=145.79.227.20:4439   , piece_id=IU5JAWLUB36JXYOUXQR7RCKFOVWUMBYR4MBOWAIMCYXOACDWUK4A, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156785699 (594h34m18.027868744s) [noise=true , address=145.79.227.19:4077   , piece_id=JMCFL7JXOQJDWBE3VAB7PPRNXCXYOZ63S42EZKJNZ2HXN6VBXLQQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156815857 (594h34m18.027898902s) [noise=true , address=145.79.227.45:4231   , piece_id=PPQOBYCHUOLIZNFCGPQ2A2U7KQLXQBK54DSNXPTV3OJAJ2ASL2BA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:19.760895014 (594h34m19.631978049s) [upload=521214]: Next returned nil segment
Dec 30 10:03:19.760911385 (594h34m19.63199444s) [upload=521214]: Waiting for error group managing segments...
Dec 30 10:03:20.148795151 (594h34m20.019878186s) [noise=true , address=145.79.227.27:4691   , piece_id=HB426ZFOAWVW7BTNBVJVMZVWLMAP45ASCCCKTR7GQ6N57LYU34VA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.149166158 (594h34m20.020249193s) [noise=true , address=145.79.227.45:4231   , piece_id=PPQOBYCHUOLIZNFCGPQ2A2U7KQLXQBK54DSNXPTV3OJAJ2ASL2BA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.149733748 (594h34m20.020816783s) [noise=true , address=145.79.227.19:4077   , piece_id=JMCFL7JXOQJDWBE3VAB7PPRNXCXYOZ63S42EZKJNZ2HXN6VBXLQQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.151892419 (594h34m20.022975454s) [noise=true , address=145.79.227.51:4410   , piece_id=UDEQMSHOZM2G46N3SCMFPSPJLBT5OISWARG7PLY4UETPC7FK3GBQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.154508520 (594h34m20.025591555s) [noise=true , address=145.79.227.39:4013   , piece_id=2JEJAAA3RALU6ZGHNFH3YIUR5TPF3OQOO73DRGNDZ5T4FU3NKTKA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.236713963 (594h34m20.107797008s) [noise=true , address=145.79.227.20:4439   , piece_id=IU5JAWLUB36JXYOUXQR7RCKFOVWUMBYR4MBOWAIMCYXOACDWUK4A, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.345793689 (594h34m20.216876724s) [noise=true , address=77.90.31.10:4518     , piece_id=YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.410251639 (594h34m20.281334684s) [noise=true , address=77.90.31.15:4700     , piece_id=D6U4KZFVXQGZBIWJ5ZXCJPVZEJ2WNVUAN4MXBSNBGHOGXVJA6LLA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.471186439 (594h34m20.342269514s) [noise=true , address=77.90.31.16:4736     , piece_id=X4SSVDEJPFV47ZJ4TPIFMJ5B4MOFWGPJWEQM5NSOJYTL7PNLSTLQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.504877693 (594h34m20.375960738s) [noise=true , address=45.91.116.8:4045     , piece_id=VIABQZSLCZ4ZYNMUH2S2YOR35DDAQUPYEGVHJLZJMMDI7CE5QZNA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.602304950 (594h34m20.473387995s) [noise=true , address=45.91.116.7:4010     , piece_id=XGNLR5TU5QE2L3H5KRYNY3E42JYWJZMLCQ7D2CURNJMLDAOV4GCQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.797462814 (594h34m20.668545859s) [noise=true , address=77.90.31.11:4555     , piece_id=N2GUODEX2INTREDGKMRLLJ7RX35F2L7YWBEBUPKGPSU3ZFR44RLA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.935458871 (594h34m20.806541916s) [noise=true , address=45.91.116.11:4119    , piece_id=5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.935488728 (594h34m20.806571783s) [seg_pos={0 0}, upload=521214]: Segment reached optimal threshold of 13 pieces.
Dec 30 10:03:20.935711993 (594h34m20.806795038s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection
Dec 30 10:03:20.935721091 (594h34m20.806804136s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936077640 (594h34m20.807160685s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF
Dec 30 10:03:20.936085134 (594h34m20.807168169s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936223698 (594h34m20.807306743s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Dec 30 10:03:20.936230561 (594h34m20.807313596s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936298570 (594h34m20.807381605s) [seg_pos={0 0}, upload=521214]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 10:03:20.936383863 (594h34m20.807466908s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 10:03:20.936391908 (594h34m20.807474953s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 10:03:21.233401234 (594h34m21.104484279s) [upload=521214]: Done uploading segments...
Dec 30 10:03:21.233425279 (594h34m21.104508324s) [upload=521214]: Upload finished. err: <nil>
Duration: 5.715944065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [.X...X...X..............X....X.X..X.X..............XX.X.XX.X]
duration: 
30.002291111s
trace: 
https://jaeger.dev.storj.io/trace/5f0586ec7b21b1c6
FAIL (2025-12-30 10:03:47 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, Piece ID: 5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ) context deadline exceeded
======LOGS======
30 10:03:17.905926101 (594h34m17.777009146s) [upload=521214]: Starting upload
Dec 30 10:03:17.905986765 (594h34m17.7770698s) [upload=521214]: Done starting upload
Dec 30 10:03:17.906714621 (594h34m17.777797657s) [upload=521214]: Uploading segments...
Dec 30 10:03:17.906764287 (594h34m17.777847342s) [upload=521214]: Got next segment. Inline: false
Dec 30 10:03:17.906822046 (594h34m17.777905091s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 10:03:17.906828439 (594h34m17.777911474s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 10:03:18.153761891 (594h34m18.024844926s) [seg_pos={0 0}, upload=521214]: Begin upload segment...
Dec 30 10:03:18.153854618 (594h34m18.024937653s) [seg_pos={0 0}, upload=521214]: Done begin upload segment.
Dec 30 10:03:18.153886619 (594h34m18.024969664s) [noise=true , address=77.90.31.15:4700     , piece_id=D6U4KZFVXQGZBIWJ5ZXCJPVZEJ2WNVUAN4MXBSNBGHOGXVJA6LLA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154059038 (594h34m18.025142083s) [noise=true , address=45.91.116.8:4045     , piece_id=VIABQZSLCZ4ZYNMUH2S2YOR35DDAQUPYEGVHJLZJMMDI7CE5QZNA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154741607 (594h34m18.025824652s) [noise=true , address=77.90.31.10:4518     , piece_id=YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.154912632 (594h34m18.025995667s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155074782 (594h34m18.026157817s) [noise=true , address=145.79.227.39:4013   , piece_id=2JEJAAA3RALU6ZGHNFH3YIUR5TPF3OQOO73DRGNDZ5T4FU3NKTKA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155294379 (594h34m18.026377424s) [noise=true , address=145.79.227.51:4410   , piece_id=UDEQMSHOZM2G46N3SCMFPSPJLBT5OISWARG7PLY4UETPC7FK3GBQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155467429 (594h34m18.026550464s) [noise=true , address=77.90.31.16:4736     , piece_id=X4SSVDEJPFV47ZJ4TPIFMJ5B4MOFWGPJWEQM5NSOJYTL7PNLSTLQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155563773 (594h34m18.026646808s) [noise=true , address=45.91.116.11:4119    , piece_id=5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155782479 (594h34m18.026865514s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.155890825 (594h34m18.02697387s) [noise=true , address=145.79.227.27:4691   , piece_id=HB426ZFOAWVW7BTNBVJVMZVWLMAP45ASCCCKTR7GQ6N57LYU34VA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156115724 (594h34m18.027198769s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156234289 (594h34m18.027317334s) [noise=true , address=77.90.31.11:4555     , piece_id=N2GUODEX2INTREDGKMRLLJ7RX35F2L7YWBEBUPKGPSU3ZFR44RLA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156488924 (594h34m18.027571959s) [noise=true , address=45.91.116.7:4010     , piece_id=XGNLR5TU5QE2L3H5KRYNY3E42JYWJZMLCQ7D2CURNJMLDAOV4GCQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156514503 (594h34m18.027597538s) [noise=true , address=145.79.227.20:4439   , piece_id=IU5JAWLUB36JXYOUXQR7RCKFOVWUMBYR4MBOWAIMCYXOACDWUK4A, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156785699 (594h34m18.027868744s) [noise=true , address=145.79.227.19:4077   , piece_id=JMCFL7JXOQJDWBE3VAB7PPRNXCXYOZ63S42EZKJNZ2HXN6VBXLQQ, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:18.156815857 (594h34m18.027898902s) [noise=true , address=145.79.227.45:4231   , piece_id=PPQOBYCHUOLIZNFCGPQ2A2U7KQLXQBK54DSNXPTV3OJAJ2ASL2BA, seg_pos={0 0}, upload=521214]: Uploading piece...
Dec 30 10:03:19.760895014 (594h34m19.631978049s) [upload=521214]: Next returned nil segment
Dec 30 10:03:19.760911385 (594h34m19.63199444s) [upload=521214]: Waiting for error group managing segments...
Dec 30 10:03:20.148795151 (594h34m20.019878186s) [noise=true , address=145.79.227.27:4691   , piece_id=HB426ZFOAWVW7BTNBVJVMZVWLMAP45ASCCCKTR7GQ6N57LYU34VA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.149166158 (594h34m20.020249193s) [noise=true , address=145.79.227.45:4231   , piece_id=PPQOBYCHUOLIZNFCGPQ2A2U7KQLXQBK54DSNXPTV3OJAJ2ASL2BA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.149733748 (594h34m20.020816783s) [noise=true , address=145.79.227.19:4077   , piece_id=JMCFL7JXOQJDWBE3VAB7PPRNXCXYOZ63S42EZKJNZ2HXN6VBXLQQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.151892419 (594h34m20.022975454s) [noise=true , address=145.79.227.51:4410   , piece_id=UDEQMSHOZM2G46N3SCMFPSPJLBT5OISWARG7PLY4UETPC7FK3GBQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.154508520 (594h34m20.025591555s) [noise=true , address=145.79.227.39:4013   , piece_id=2JEJAAA3RALU6ZGHNFH3YIUR5TPF3OQOO73DRGNDZ5T4FU3NKTKA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.236713963 (594h34m20.107797008s) [noise=true , address=145.79.227.20:4439   , piece_id=IU5JAWLUB36JXYOUXQR7RCKFOVWUMBYR4MBOWAIMCYXOACDWUK4A, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.345793689 (594h34m20.216876724s) [noise=true , address=77.90.31.10:4518     , piece_id=YI5V35GXTEEWW56GQG4ULDVG2326E5JPA2PQVOJD26HDSUXWQZCQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.410251639 (594h34m20.281334684s) [noise=true , address=77.90.31.15:4700     , piece_id=D6U4KZFVXQGZBIWJ5ZXCJPVZEJ2WNVUAN4MXBSNBGHOGXVJA6LLA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.471186439 (594h34m20.342269514s) [noise=true , address=77.90.31.16:4736     , piece_id=X4SSVDEJPFV47ZJ4TPIFMJ5B4MOFWGPJWEQM5NSOJYTL7PNLSTLQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.504877693 (594h34m20.375960738s) [noise=true , address=45.91.116.8:4045     , piece_id=VIABQZSLCZ4ZYNMUH2S2YOR35DDAQUPYEGVHJLZJMMDI7CE5QZNA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.602304950 (594h34m20.473387995s) [noise=true , address=45.91.116.7:4010     , piece_id=XGNLR5TU5QE2L3H5KRYNY3E42JYWJZMLCQ7D2CURNJMLDAOV4GCQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.797462814 (594h34m20.668545859s) [noise=true , address=77.90.31.11:4555     , piece_id=N2GUODEX2INTREDGKMRLLJ7RX35F2L7YWBEBUPKGPSU3ZFR44RLA, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.935458871 (594h34m20.806541916s) [noise=true , address=45.91.116.11:4119    , piece_id=5URCOKJNJTGBTVPLEAZJ7XVYZ46NY7XS5TJ5LKZ6FPUAXJZQQUPQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: <nil>
Dec 30 10:03:20.935488728 (594h34m20.806571783s) [seg_pos={0 0}, upload=521214]: Segment reached optimal threshold of 13 pieces.
Dec 30 10:03:20.935711993 (594h34m20.806795038s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33998->45.91.116.5:4298: use of closed network connection
Dec 30 10:03:20.935721091 (594h34m20.806804136s) [noise=true , address=45.91.116.5:4298     , piece_id=7S4KXBHF4TU3EH4DVESW5YULYOAOEVTCJVR23UK4YWOAGXC2IF5Q, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936077640 (594h34m20.807160685s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF
Dec 30 10:03:20.936085134 (594h34m20.807168169s) [noise=true , address=77.90.31.19:4842     , piece_id=PN2LJKSSB5GV5AM5GJYOGODY4WRD4W7U6TDI5UBVMPB36CU5VSXQ, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936223698 (594h34m20.807306743s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Dec 30 10:03:20.936230561 (594h34m20.807313596s) [noise=true , address=45.91.116.10:4083    , piece_id=HVQWAWKPJAASHILVC2EIFBG2XAYHKA6GA2AF6HMMXK3JCBMUTV7Q, seg_pos={0 0}, upload=521214]: Long tail cancellation due to optimal threshold
Dec 30 10:03:20.936298570 (594h34m20.807381605s) [seg_pos={0 0}, upload=521214]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 10:03:20.936383863 (594h34m20.807466908s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 10:03:20.936391908 (594h34m20.807474953s) [upload=521214]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 10:03:21.233401234 (594h34m21.104484279s) [upload=521214]: Done uploading segments...
Dec 30 10:03:21.233425279 (594h34m21.104508324s) [upload=521214]: Upload finished. err: <nil>
Duration: 30.002304064s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [X........................................X..................]
duration: 
44.83403857s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:03:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 44.834053839s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [....XXX..X.X.X.......XX....X..X.XX.........X..X......X......]
duration: 
3.572324389s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:03:27 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.572349657s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [X....XXX..X.X.X.......XX....X..X.XX.........X..X......X.....]
duration: 
3.557182216s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:03:00 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.557214427s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.X........................................X.................]
duration: 
35.649879736s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:02:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 35.649905085s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [..X...X...X..............X....X.X..X.X..............XX.X.XX.]
duration: 
6.173542898s
trace: 
https://jaeger.dev.storj.io/trace/4601781ec25671e9
FAIL (2025-12-30 10:02:45 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ) context deadline exceeded; piecestore: (Node ID: 12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, Piece ID: SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA) context deadline exceeded
======LOGS======
30 10:01:27.395569412 (594h32m27.266652457s) [upload=521189]: Starting upload
Dec 30 10:01:27.395699791 (594h32m27.266782836s) [upload=521189]: Done starting upload
Dec 30 10:01:27.395802105 (594h32m27.26688514s) [upload=521189]: Uploading segments...
Dec 30 10:01:27.396081558 (594h32m27.267164603s) [upload=521189]: Got next segment. Inline: false
Dec 30 10:01:27.396117656 (594h32m27.267200701s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 10:01:27.396126022 (594h32m27.267209057s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 10:01:27.643667022 (594h32m27.514750057s) [seg_pos={0 0}, upload=521189]: Begin upload segment...
Dec 30 10:01:27.643738107 (594h32m27.514821142s) [seg_pos={0 0}, upload=521189]: Done begin upload segment.
Dec 30 10:01:27.643810655 (594h32m27.51489369s) [noise=true , address=145.79.227.37:4566   , piece_id=G4GJLXI6F5ZKOY3UNCQUGSOVSJR6S5FZZIK3RSFSIU3DYIIC5HPQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.643822588 (594h32m27.514905623s) [noise=true , address=77.90.31.10:4518     , piece_id=GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644344602 (594h32m27.515427637s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644354160 (594h32m27.515437205s) [noise=true , address=145.79.227.9:4365    , piece_id=NOT7AJOZAC5WWIVXBQU6XIHICVI62LWRSKWPYMI6UZWKYRKYOXSA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644395238 (594h32m27.515478273s) [noise=true , address=45.91.116.9:4262     , piece_id=SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644669791 (594h32m27.515752826s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644721830 (594h32m27.515804875s) [noise=true , address=145.79.227.41:4085   , piece_id=C3AP4HMCRPVJTZC2B2WGZFKO2NYAV7IH6BGJV72ZULFREHD7D4GQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644922703 (594h32m27.516005738s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.645228595 (594h32m27.51631164s) [noise=true , address=145.79.227.10:4402   , piece_id=O6V5OL7VG5PT7NAXZXXEX2S3MAOLO3ELNUBL2Z42KZ7ONT6PKKXQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.645677480 (594h32m27.516760515s) [noise=true , address=145.79.227.36:4517   , piece_id=TQVFCLP3KLJ7BFP6R5CCOVY4YEAEYQGRRTMIPEVKR7EBIOMY6UQA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646149348 (594h32m27.517232393s) [noise=true , address=145.79.227.45:4230   , piece_id=5WAICLHPL76JN6FQDDSACCWNSLUHHFV4RXGB7RTFUSJIKDJBBMGA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646501389 (594h32m27.517584424s) [noise=true , address=77.90.31.17:4770     , piece_id=WW2R3O77DIGU2SVEONCTQ4PXQJNTJLFD5N3KTVFSYYEM43D73DIQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646864450 (594h32m27.517947495s) [noise=true , address=145.79.227.15:4876   , piece_id=4TOVML42S2GDOK6LIXLK7XIDX4O6H4YYRDHK3PQXPZ6WKIM3FFGQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.647183337 (594h32m27.518266372s) [noise=true , address=145.79.227.13:4802   , piece_id=DJJWA6QKCJ7IOEGMFIW5VEB3C37WQDOSMAQ7PLYNHXYFOX3ROVJQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.647567178 (594h32m27.518650223s) [noise=true , address=145.79.227.3:4149    , piece_id=SZRHTINVHGU6MGPSF374DE3QUUPZU7TTXX3EMMLMS6GXH4JM6E2A, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.652435498 (594h32m27.523518533s) [noise=true , address=145.79.227.29:4941   , piece_id=JFK3GCEK5VUOTES7F6EVXZCRU5KWLVBBDBFDDLEBXUEUFBCINEDA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:29.240174802 (594h32m29.111257837s) [upload=521189]: Next returned nil segment
Dec 30 10:01:29.240190721 (594h32m29.111273767s) [upload=521189]: Waiting for error group managing segments...
Dec 30 10:01:29.621825439 (594h32m29.492908474s) [noise=true , address=145.79.227.15:4876   , piece_id=4TOVML42S2GDOK6LIXLK7XIDX4O6H4YYRDHK3PQXPZ6WKIM3FFGQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.637191519 (594h32m29.508274554s) [noise=true , address=145.79.227.36:4517   , piece_id=TQVFCLP3KLJ7BFP6R5CCOVY4YEAEYQGRRTMIPEVKR7EBIOMY6UQA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.642098893 (594h32m29.513181929s) [noise=true , address=145.79.227.9:4365    , piece_id=NOT7AJOZAC5WWIVXBQU6XIHICVI62LWRSKWPYMI6UZWKYRKYOXSA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.654506780 (594h32m29.525589825s) [noise=true , address=145.79.227.13:4802   , piece_id=DJJWA6QKCJ7IOEGMFIW5VEB3C37WQDOSMAQ7PLYNHXYFOX3ROVJQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.655492577 (594h32m29.526575612s) [noise=true , address=145.79.227.45:4230   , piece_id=5WAICLHPL76JN6FQDDSACCWNSLUHHFV4RXGB7RTFUSJIKDJBBMGA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.672519529 (594h32m29.543602574s) [noise=true , address=145.79.227.10:4402   , piece_id=O6V5OL7VG5PT7NAXZXXEX2S3MAOLO3ELNUBL2Z42KZ7ONT6PKKXQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.711474095 (594h32m29.58255714s) [noise=true , address=145.79.227.41:4085   , piece_id=C3AP4HMCRPVJTZC2B2WGZFKO2NYAV7IH6BGJV72ZULFREHD7D4GQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.718179213 (594h32m29.589262258s) [noise=true , address=145.79.227.37:4566   , piece_id=G4GJLXI6F5ZKOY3UNCQUGSOVSJR6S5FZZIK3RSFSIU3DYIIC5HPQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.721606318 (594h32m29.592689353s) [noise=true , address=145.79.227.3:4149    , piece_id=SZRHTINVHGU6MGPSF374DE3QUUPZU7TTXX3EMMLMS6GXH4JM6E2A, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.810294887 (594h32m29.681377922s) [noise=true , address=77.90.31.17:4770     , piece_id=WW2R3O77DIGU2SVEONCTQ4PXQJNTJLFD5N3KTVFSYYEM43D73DIQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.893401907 (594h32m29.764484942s) [noise=true , address=145.79.227.29:4941   , piece_id=JFK3GCEK5VUOTES7F6EVXZCRU5KWLVBBDBFDDLEBXUEUFBCINEDA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.950187466 (594h32m29.821270501s) [noise=true , address=77.90.31.10:4518     , piece_id=GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:30.007829636 (594h32m29.878912681s) [noise=true , address=45.91.116.9:4262     , piece_id=SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:30.007853562 (594h32m29.878936597s) [seg_pos={0 0}, upload=521189]: Segment reached optimal threshold of 13 pieces.
Dec 30 10:01:30.008091835 (594h32m29.87917489s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF
Dec 30 10:01:30.008102075 (594h32m29.87918512s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008258823 (594h32m29.879341868s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: expected piece hash; context canceled; EOF
Dec 30 10:01:30.008271417 (594h32m29.879354452s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008329016 (594h32m29.879412051s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection
Dec 30 10:01:30.008336702 (594h32m29.879419737s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008358012 (594h32m29.879441057s) [seg_pos={0 0}, upload=521189]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 10:01:30.008485244 (594h32m29.879568279s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 10:01:30.008492448 (594h32m29.879575493s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 10:01:30.303861961 (594h32m30.174944996s) [upload=521189]: Done uploading segments...
Dec 30 10:01:30.303892510 (594h32m30.174975545s) [upload=521189]: Upload finished. err: <nil>
Duration: 6.173568536s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [.X....XXX..X.X.X.......XX....X..X.XX.........X..X......X....]
duration: 
3.466372706s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:02:00 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.466554633s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [...X...X...X..............X....X.X..X.X..............XX.X.XX]
duration: 
30.003471149s
trace: 
https://jaeger.dev.storj.io/trace/4601781ec25671e9
FAIL (2025-12-30 10:01:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ) context deadline exceeded; piecestore: (Node ID: 12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, Piece ID: SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA) context deadline exceeded
======LOGS======
30 10:01:27.395569412 (594h32m27.266652457s) [upload=521189]: Starting upload
Dec 30 10:01:27.395699791 (594h32m27.266782836s) [upload=521189]: Done starting upload
Dec 30 10:01:27.395802105 (594h32m27.26688514s) [upload=521189]: Uploading segments...
Dec 30 10:01:27.396081558 (594h32m27.267164603s) [upload=521189]: Got next segment. Inline: false
Dec 30 10:01:27.396117656 (594h32m27.267200701s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 10:01:27.396126022 (594h32m27.267209057s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 10:01:27.643667022 (594h32m27.514750057s) [seg_pos={0 0}, upload=521189]: Begin upload segment...
Dec 30 10:01:27.643738107 (594h32m27.514821142s) [seg_pos={0 0}, upload=521189]: Done begin upload segment.
Dec 30 10:01:27.643810655 (594h32m27.51489369s) [noise=true , address=145.79.227.37:4566   , piece_id=G4GJLXI6F5ZKOY3UNCQUGSOVSJR6S5FZZIK3RSFSIU3DYIIC5HPQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.643822588 (594h32m27.514905623s) [noise=true , address=77.90.31.10:4518     , piece_id=GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644344602 (594h32m27.515427637s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644354160 (594h32m27.515437205s) [noise=true , address=145.79.227.9:4365    , piece_id=NOT7AJOZAC5WWIVXBQU6XIHICVI62LWRSKWPYMI6UZWKYRKYOXSA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644395238 (594h32m27.515478273s) [noise=true , address=45.91.116.9:4262     , piece_id=SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644669791 (594h32m27.515752826s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644721830 (594h32m27.515804875s) [noise=true , address=145.79.227.41:4085   , piece_id=C3AP4HMCRPVJTZC2B2WGZFKO2NYAV7IH6BGJV72ZULFREHD7D4GQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.644922703 (594h32m27.516005738s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.645228595 (594h32m27.51631164s) [noise=true , address=145.79.227.10:4402   , piece_id=O6V5OL7VG5PT7NAXZXXEX2S3MAOLO3ELNUBL2Z42KZ7ONT6PKKXQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.645677480 (594h32m27.516760515s) [noise=true , address=145.79.227.36:4517   , piece_id=TQVFCLP3KLJ7BFP6R5CCOVY4YEAEYQGRRTMIPEVKR7EBIOMY6UQA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646149348 (594h32m27.517232393s) [noise=true , address=145.79.227.45:4230   , piece_id=5WAICLHPL76JN6FQDDSACCWNSLUHHFV4RXGB7RTFUSJIKDJBBMGA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646501389 (594h32m27.517584424s) [noise=true , address=77.90.31.17:4770     , piece_id=WW2R3O77DIGU2SVEONCTQ4PXQJNTJLFD5N3KTVFSYYEM43D73DIQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.646864450 (594h32m27.517947495s) [noise=true , address=145.79.227.15:4876   , piece_id=4TOVML42S2GDOK6LIXLK7XIDX4O6H4YYRDHK3PQXPZ6WKIM3FFGQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.647183337 (594h32m27.518266372s) [noise=true , address=145.79.227.13:4802   , piece_id=DJJWA6QKCJ7IOEGMFIW5VEB3C37WQDOSMAQ7PLYNHXYFOX3ROVJQ, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.647567178 (594h32m27.518650223s) [noise=true , address=145.79.227.3:4149    , piece_id=SZRHTINVHGU6MGPSF374DE3QUUPZU7TTXX3EMMLMS6GXH4JM6E2A, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:27.652435498 (594h32m27.523518533s) [noise=true , address=145.79.227.29:4941   , piece_id=JFK3GCEK5VUOTES7F6EVXZCRU5KWLVBBDBFDDLEBXUEUFBCINEDA, seg_pos={0 0}, upload=521189]: Uploading piece...
Dec 30 10:01:29.240174802 (594h32m29.111257837s) [upload=521189]: Next returned nil segment
Dec 30 10:01:29.240190721 (594h32m29.111273767s) [upload=521189]: Waiting for error group managing segments...
Dec 30 10:01:29.621825439 (594h32m29.492908474s) [noise=true , address=145.79.227.15:4876   , piece_id=4TOVML42S2GDOK6LIXLK7XIDX4O6H4YYRDHK3PQXPZ6WKIM3FFGQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.637191519 (594h32m29.508274554s) [noise=true , address=145.79.227.36:4517   , piece_id=TQVFCLP3KLJ7BFP6R5CCOVY4YEAEYQGRRTMIPEVKR7EBIOMY6UQA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.642098893 (594h32m29.513181929s) [noise=true , address=145.79.227.9:4365    , piece_id=NOT7AJOZAC5WWIVXBQU6XIHICVI62LWRSKWPYMI6UZWKYRKYOXSA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.654506780 (594h32m29.525589825s) [noise=true , address=145.79.227.13:4802   , piece_id=DJJWA6QKCJ7IOEGMFIW5VEB3C37WQDOSMAQ7PLYNHXYFOX3ROVJQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.655492577 (594h32m29.526575612s) [noise=true , address=145.79.227.45:4230   , piece_id=5WAICLHPL76JN6FQDDSACCWNSLUHHFV4RXGB7RTFUSJIKDJBBMGA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.672519529 (594h32m29.543602574s) [noise=true , address=145.79.227.10:4402   , piece_id=O6V5OL7VG5PT7NAXZXXEX2S3MAOLO3ELNUBL2Z42KZ7ONT6PKKXQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.711474095 (594h32m29.58255714s) [noise=true , address=145.79.227.41:4085   , piece_id=C3AP4HMCRPVJTZC2B2WGZFKO2NYAV7IH6BGJV72ZULFREHD7D4GQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.718179213 (594h32m29.589262258s) [noise=true , address=145.79.227.37:4566   , piece_id=G4GJLXI6F5ZKOY3UNCQUGSOVSJR6S5FZZIK3RSFSIU3DYIIC5HPQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.721606318 (594h32m29.592689353s) [noise=true , address=145.79.227.3:4149    , piece_id=SZRHTINVHGU6MGPSF374DE3QUUPZU7TTXX3EMMLMS6GXH4JM6E2A, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.810294887 (594h32m29.681377922s) [noise=true , address=77.90.31.17:4770     , piece_id=WW2R3O77DIGU2SVEONCTQ4PXQJNTJLFD5N3KTVFSYYEM43D73DIQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.893401907 (594h32m29.764484942s) [noise=true , address=145.79.227.29:4941   , piece_id=JFK3GCEK5VUOTES7F6EVXZCRU5KWLVBBDBFDDLEBXUEUFBCINEDA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:29.950187466 (594h32m29.821270501s) [noise=true , address=77.90.31.10:4518     , piece_id=GCJBM237LJ2NN5F7XDDKKI2775DOBSCFOFOFASUOKLEVLN7OM7WQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:30.007829636 (594h32m29.878912681s) [noise=true , address=45.91.116.9:4262     , piece_id=SDYPOOLXHMCOLDEWQ3GFZ5JMJM4OIQTOMENA7IBJE6XE7EVBJUBA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: <nil>
Dec 30 10:01:30.007853562 (594h32m29.878936597s) [seg_pos={0 0}, upload=521189]: Segment reached optimal threshold of 13 pieces.
Dec 30 10:01:30.008091835 (594h32m29.87917489s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF
Dec 30 10:01:30.008102075 (594h32m29.87918512s) [noise=true , address=145.79.227.5:4222    , piece_id=CPGUYJRQFUFME7MSWI427USQXAFTGBKF5G6UFXKKGK7DTMEUXNRA, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008258823 (594h32m29.879341868s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: expected piece hash; context canceled; EOF
Dec 30 10:01:30.008271417 (594h32m29.879354452s) [noise=true , address=145.79.227.43:4159   , piece_id=JUADDZK4Y4C5JXO6Y3F42UUWREFBAD5IYSHI447PTD5R3VTFXC2Q, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008329016 (594h32m29.879412051s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52474->45.91.116.11:4119: use of closed network connection
Dec 30 10:01:30.008336702 (594h32m29.879419737s) [noise=true , address=45.91.116.11:4119    , piece_id=QAWH6H4LDJS64JPLGYNI55DTVQKP3L2XVIXLNWTZERY5UVYZTTKQ, seg_pos={0 0}, upload=521189]: Long tail cancellation due to optimal threshold
Dec 30 10:01:30.008358012 (594h32m29.879441057s) [seg_pos={0 0}, upload=521189]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 10:01:30.008485244 (594h32m29.879568279s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 10:01:30.008492448 (594h32m29.879575493s) [upload=521189]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 10:01:30.303861961 (594h32m30.174944996s) [upload=521189]: Done uploading segments...
Dec 30 10:01:30.303892510 (594h32m30.174975545s) [upload=521189]: Upload finished. err: <nil>
Duration: 30.003493261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..X........................................X................]
duration: 
46.485648772s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:01:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 46.485670854s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [X...X...X...X..............X....X.X..X.X..............XX.X.X]
duration: 
30.004321705s
trace: 
https://jaeger.dev.storj.io/trace/1ded36620493f86b
FAIL (2025-12-30 10:01:16 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
c 30 10:00:46.736574436 (594h31m46.607657471s) [upload=521178]: Starting upload
Dec 30 10:00:46.736666160 (594h31m46.607749205s) [upload=521178]: Done starting upload
Dec 30 10:00:46.736804794 (594h31m46.607887849s) [upload=521178]: Uploading segments...
Dec 30 10:00:46.736942997 (594h31m46.608026032s) [upload=521178]: Got next segment. Inline: false
Dec 30 10:00:46.737009423 (594h31m46.608092468s) [upload=521178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 10:00:46.737016597 (594h31m46.608099632s) [upload=521178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 10:00:46.980301573 (594h31m46.851384618s) [seg_pos={0 0}, upload=521178]: Begin upload segment...
Dec 30 10:00:46.980399910 (594h31m46.851482955s) [seg_pos={0 0}, upload=521178]: Done begin upload segment.
Dec 30 10:00:46.980438734 (594h31m46.851521779s) [noise=true , address=145.79.227.48:4302   , piece_id=N2BTJMVYUIM5NUFBWOWF46P36SEET2OWCBFVRFQW7PUFNCETJFNQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.980535679 (594h31m46.851618724s) [noise=true , address=145.79.227.14:4837   , piece_id=UFHQJ3F3PUNMLCSXVZKRT5LEMR6VFIWLOH7EJ3XBEOUKNNCPA2EA, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.981280807 (594h31m46.852363852s) [noise=true , address=145.79.227.36:4518   , piece_id=7ONG4GHQMBOZYQ4TRG4EOKTWNMGXTURWSEDILKEDGBU7NVRAPOKQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.981674957 (594h31m46.852757992s) [noise=true , address=145.79.227.11:4731   , piece_id=UPQDJ43EA64K7NW7NX3WMJHXDDRWV7JOTIIXOQKJBKCJXXC26UJQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.982269510 (594h31m46.853352555s) [noise=true , address=77.90.31.14:4663     , piece_id=25RW47LJ45ENFQ5IJ42WS55C6S57NHPHOJZMDALKCGYNNUTRDPSQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.982657780 (594h31m46.853740815s) [noise=true , address=145.79.227.38:4979   , piece_id=CCENL4EIYVBDFMY3PUCMGGXFFKNCFGIDPVDCYSOJY6LGWCBHROLQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.983055777 (594h31m46.854138822s) [noise=true , address=77.90.31.11:4554     , piece_id=7XDHAWCRTABWPM4NQJX5JXIH4MPWF76HZDBYLFKEDHF7RD3KCWVQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.983549046 (594h31m46.854632091s) [noise=true , address=145.79.227.28:4726   , piece_id=VONBLJUEW4HWHJRUH2VTE3GHGOV7GXEQ7TY2BVZSA5YDM5RPQQTA, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.983974407 (594h31m46.855057442s) [noise=true , address=145.79.227.39:4013   , piece_id=SSFVC23ERRI6CL77KDBN7VWORWXEPUCL5QGXM4RFU5HMJKJ7XXPQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.984211167 (594h31m46.855294212s) [noise=true , address=77.90.31.17:4770     , piece_id=WKG62D4UDBKKDB74YU5MKWZLOAD3TMVNA3LFYUPDUPDO7IT7PRUQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.984582985 (594h31m46.85566603s) [noise=true , address=45.91.116.4:4940     , piece_id=UBKP3DOO563WY2ESYX3K6GQSP66FK6CHHWZR6PCEFYNGBE6W5YOQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.984983278 (594h31m46.856066313s) [noise=true , address=145.79.227.16:4911   , piece_id=XJYMR3QRJRYKJK3FXU4WPYZD6KHY6CPIABGESD7XMLIGJ7JZZQTA, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.985377348 (594h31m46.856460393s) [noise=true , address=45.91.116.11:4118    , piece_id=GJGJRIODN5MZPY66QCUUFER6X3D3DP5CL4II7NUJV4RE5GURPGUQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.985756510 (594h31m46.856839555s) [noise=true , address=145.79.227.2:4114    , piece_id=QNRWDIADV5D443OIBTIGFWBP36ZTJT23FQ7OWCNCXOK7YUQNO5KQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.986201056 (594h31m46.857284091s) [noise=true , address=145.79.227.3:4151    , piece_id=HNOQXPG2MX4APLDIVNLBU22SHA3V7CGZDLIERURURT77QC4WMY4A, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:46.986517279 (594h31m46.857600314s) [noise=true , address=145.79.227.43:4157   , piece_id=QQRNB4PWXRCM6ZWYVZYOA3V4IMQWHXPC57AK4XRBWRZDPJCASCCQ, seg_pos={0 0}, upload=521178]: Uploading piece...
Dec 30 10:00:48.591135511 (594h31m48.462218547s) [upload=521178]: Next returned nil segment
Dec 30 10:00:48.591161491 (594h31m48.462244526s) [upload=521178]: Waiting for error group managing segments...
Dec 30 10:00:48.974164444 (594h31m48.845247479s) [noise=true , address=145.79.227.14:4837   , piece_id=UFHQJ3F3PUNMLCSXVZKRT5LEMR6VFIWLOH7EJ3XBEOUKNNCPA2EA, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.976369963 (594h31m48.847452998s) [noise=true , address=145.79.227.39:4013   , piece_id=SSFVC23ERRI6CL77KDBN7VWORWXEPUCL5QGXM4RFU5HMJKJ7XXPQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.978774291 (594h31m48.849857326s) [noise=true , address=145.79.227.28:4726   , piece_id=VONBLJUEW4HWHJRUH2VTE3GHGOV7GXEQ7TY2BVZSA5YDM5RPQQTA, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.979147963 (594h31m48.850230998s) [noise=true , address=145.79.227.16:4911   , piece_id=XJYMR3QRJRYKJK3FXU4WPYZD6KHY6CPIABGESD7XMLIGJ7JZZQTA, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.989424060 (594h31m48.860507105s) [noise=true , address=145.79.227.2:4114    , piece_id=QNRWDIADV5D443OIBTIGFWBP36ZTJT23FQ7OWCNCXOK7YUQNO5KQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.992504545 (594h31m48.86358758s) [noise=true , address=145.79.227.36:4518   , piece_id=7ONG4GHQMBOZYQ4TRG4EOKTWNMGXTURWSEDILKEDGBU7NVRAPOKQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.996042252 (594h31m48.867125297s) [noise=true , address=145.79.227.48:4302   , piece_id=N2BTJMVYUIM5NUFBWOWF46P36SEET2OWCBFVRFQW7PUFNCETJFNQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:48.997155221 (594h31m48.868238267s) [noise=true , address=145.79.227.3:4151    , piece_id=HNOQXPG2MX4APLDIVNLBU22SHA3V7CGZDLIERURURT77QC4WMY4A, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.053745039 (594h31m48.924828074s) [noise=true , address=145.79.227.11:4731   , piece_id=UPQDJ43EA64K7NW7NX3WMJHXDDRWV7JOTIIXOQKJBKCJXXC26UJQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.060008746 (594h31m48.931091781s) [noise=true , address=145.79.227.38:4979   , piece_id=CCENL4EIYVBDFMY3PUCMGGXFFKNCFGIDPVDCYSOJY6LGWCBHROLQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.169033367 (594h31m49.040116402s) [noise=true , address=77.90.31.11:4554     , piece_id=7XDHAWCRTABWPM4NQJX5JXIH4MPWF76HZDBYLFKEDHF7RD3KCWVQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.246116336 (594h31m49.117199381s) [noise=true , address=77.90.31.17:4770     , piece_id=WKG62D4UDBKKDB74YU5MKWZLOAD3TMVNA3LFYUPDUPDO7IT7PRUQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.416233477 (594h31m49.287316512s) [noise=true , address=45.91.116.4:4940     , piece_id=UBKP3DOO563WY2ESYX3K6GQSP66FK6CHHWZR6PCEFYNGBE6W5YOQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: <nil>
Dec 30 10:00:49.416252232 (594h31m49.287335267s) [seg_pos={0 0}, upload=521178]: Segment reached optimal threshold of 13 pieces.
Dec 30 10:00:49.416833229 (594h31m49.287916264s) [noise=true , address=77.90.31.14:4663     , piece_id=25RW47LJ45ENFQ5IJ42WS55C6S57NHPHOJZMDALKCGYNNUTRDPSQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:39738->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:39738->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39738->77.90.31.14:4663: use of closed network connection
Dec 30 10:00:49.416841544 (594h31m49.28792459s) [noise=true , address=77.90.31.14:4663     , piece_id=25RW47LJ45ENFQ5IJ42WS55C6S57NHPHOJZMDALKCGYNNUTRDPSQ, seg_pos={0 0}, upload=521178]: Long tail cancellation due to optimal threshold
Dec 30 10:00:49.417079057 (594h31m49.288162092s) [noise=true , address=45.91.116.11:4118    , piece_id=GJGJRIODN5MZPY66QCUUFER6X3D3DP5CL4II7NUJV4RE5GURPGUQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Dec 30 10:00:49.417085108 (594h31m49.288168143s) [noise=true , address=45.91.116.11:4118    , piece_id=GJGJRIODN5MZPY66QCUUFER6X3D3DP5CL4II7NUJV4RE5GURPGUQ, seg_pos={0 0}, upload=521178]: Long tail cancellation due to optimal threshold
Dec 30 10:00:49.417261755 (594h31m49.28834479s) [noise=true , address=145.79.227.43:4157   , piece_id=QQRNB4PWXRCM6ZWYVZYOA3V4IMQWHXPC57AK4XRBWRZDPJCASCCQ, seg_pos={0 0}, upload=521178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: expected piece hash; context canceled; EOF
Dec 30 10:00:49.417267646 (594h31m49.288350681s) [noise=true , address=145.79.227.43:4157   , piece_id=QQRNB4PWXRCM6ZWYVZYOA3V4IMQWHXPC57AK4XRBWRZDPJCASCCQ, seg_pos={0 0}, upload=521178]: Long tail cancellation due to optimal threshold
Dec 30 10:00:49.417283987 (594h31m49.288367022s) [seg_pos={0 0}, upload=521178]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 10:00:49.417354091 (594h31m49.288437126s) [upload=521178]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 10:00:49.417360302 (594h31m49.288443337s) [upload=521178]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 10:00:49.713139937 (594h31m49.584222972s) [upload=521178]: Done uploading segments...
Dec 30 10:00:49.713159475 (594h31m49.58424252s) [upload=521178]: Upload finished. err: <nil>
Duration: 30.004348304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...X........................................X...............]
duration: 
39.683047003s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:01:10 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 39.683077361s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [..X....XXX..X.X.X.......XX....X..X.XX.........X..X......X...]
duration: 
5.003257446s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:01:03 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.003324192s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [...X....XXX..X.X.X.......XX....X..X.XX.........X..X......X..]
duration: 
3.450532746s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:00:27 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.450561009s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [.X...X...X...X..............X....X.X..X.X..............XX.X.]
duration: 
5.969942598s
trace: 
https://jaeger.dev.storj.io/trace/56455cb83c53e4bf
FAIL (2025-12-30 10:00:25 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, Piece ID: V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA) context deadline exceeded; piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA) context deadline exceeded
======LOGS======
30 09:59:15.268425076 (594h30m15.139508121s) [upload=521156]: Starting upload
Dec 30 09:59:15.268521350 (594h30m15.139604385s) [upload=521156]: Done starting upload
Dec 30 09:59:15.268677877 (594h30m15.139760922s) [upload=521156]: Uploading segments...
Dec 30 09:59:15.268960014 (594h30m15.140043049s) [upload=521156]: Got next segment. Inline: false
Dec 30 09:59:15.269045027 (594h30m15.140128072s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:59:15.269052902 (594h30m15.140135957s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:59:15.514504773 (594h30m15.385587808s) [seg_pos={0 0}, upload=521156]: Begin upload segment...
Dec 30 09:59:15.514555731 (594h30m15.385638766s) [seg_pos={0 0}, upload=521156]: Done begin upload segment.
Dec 30 09:59:15.514594785 (594h30m15.38567783s) [noise=true , address=145.79.227.13:4804   , piece_id=VKWWH2L55TPWU3SFUGXPK3SIKYUPP4NOMFL4Z6O6QPIOHM45WREA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.514613640 (594h30m15.385696686s) [noise=true , address=145.79.227.5:4221    , piece_id=BVGPFMIU2FKB3ODUCRFOL3MJGOJGCL2WGQM36YSNB55BVXPVYR5A, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515148249 (594h30m15.386231284s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515403585 (594h30m15.38648663s) [noise=true , address=145.79.227.36:4517   , piece_id=GJJK3NBZWQYDURWGZ3QMBIVYURJ2AIZYYZ3OJEDB6JRRSFU3DN6Q, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515704488 (594h30m15.386787523s) [noise=true , address=45.91.116.8:4047     , piece_id=V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515994029 (594h30m15.387077064s) [noise=true , address=145.79.227.14:4840   , piece_id=OFKL6PZSWNLAI6S7RNHQE23S4RN5WPSPAQBXHMZR3MNOV2TYBW6A, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516246921 (594h30m15.387329966s) [noise=true , address=145.79.227.37:4566   , piece_id=BHKJQBDJHZ2NCPAR2Q37ZWVEF2XUDDLCCZZXRKHUCDQFY57HASBA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516610383 (594h30m15.387693418s) [noise=true , address=45.91.116.9:4263     , piece_id=IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516924732 (594h30m15.388007767s) [noise=true , address=145.79.227.18:4006   , piece_id=QSSIOL3KRU5CACHELJ7QSXOW4CY6UX2PBS6AP5ENFFG4B257FQOQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.517179366 (594h30m15.388262401s) [noise=true , address=77.90.31.14:4662     , piece_id=3CYNHYEFUEFWAY6SJA4ENNMNF22AKILNUW7N6DEP7X3ZUN7ZY6HQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.517454601 (594h30m15.388537646s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518098957 (594h30m15.389181992s) [noise=true , address=145.79.227.41:4086   , piece_id=A6LE5QCA2YOC25K67X65IDZOGEH46BUUNQDEBXIE37JOZH7ZI42Q, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518550056 (594h30m15.389633101s) [noise=true , address=145.79.227.4:4187    , piece_id=KOV27O254S6MB6EJBFTW7MZHJJENSFKQFA5NFHOBELOBYLSK5PSA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518925772 (594h30m15.390008807s) [noise=true , address=145.79.227.51:4410   , piece_id=CVJJR5M7CUXOY74ZCFC2BXLFUQORF57YXLWL6EIT7IAYP5VB5ZZA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.519337927 (594h30m15.390420962s) [noise=true , address=145.79.227.38:4978   , piece_id=J6G5HBREOULMW4X74BO6SJR6PXZ5SKXLBMUIB5IKTQP4F5VK24DQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.521755059 (594h30m15.392838104s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:17.214727013 (594h30m17.085810048s) [upload=521156]: Next returned nil segment
Dec 30 09:59:17.214738596 (594h30m17.085821631s) [upload=521156]: Waiting for error group managing segments...
Dec 30 09:59:17.497607165 (594h30m17.368690201s) [noise=true , address=145.79.227.18:4006   , piece_id=QSSIOL3KRU5CACHELJ7QSXOW4CY6UX2PBS6AP5ENFFG4B257FQOQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.497722024 (594h30m17.368805059s) [noise=true , address=145.79.227.14:4840   , piece_id=OFKL6PZSWNLAI6S7RNHQE23S4RN5WPSPAQBXHMZR3MNOV2TYBW6A, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.501187864 (594h30m17.372270899s) [noise=true , address=145.79.227.36:4517   , piece_id=GJJK3NBZWQYDURWGZ3QMBIVYURJ2AIZYYZ3OJEDB6JRRSFU3DN6Q, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.503909276 (594h30m17.374992321s) [noise=true , address=145.79.227.5:4221    , piece_id=BVGPFMIU2FKB3ODUCRFOL3MJGOJGCL2WGQM36YSNB55BVXPVYR5A, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.511872969 (594h30m17.382956004s) [noise=true , address=145.79.227.51:4410   , piece_id=CVJJR5M7CUXOY74ZCFC2BXLFUQORF57YXLWL6EIT7IAYP5VB5ZZA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.515389095 (594h30m17.38647214s) [noise=true , address=145.79.227.38:4978   , piece_id=J6G5HBREOULMW4X74BO6SJR6PXZ5SKXLBMUIB5IKTQP4F5VK24DQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.539482172 (594h30m17.410565217s) [noise=true , address=145.79.227.4:4187    , piece_id=KOV27O254S6MB6EJBFTW7MZHJJENSFKQFA5NFHOBELOBYLSK5PSA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.584163543 (594h30m17.455246578s) [noise=true , address=145.79.227.37:4566   , piece_id=BHKJQBDJHZ2NCPAR2Q37ZWVEF2XUDDLCCZZXRKHUCDQFY57HASBA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.593999793 (594h30m17.465082828s) [noise=true , address=145.79.227.41:4086   , piece_id=A6LE5QCA2YOC25K67X65IDZOGEH46BUUNQDEBXIE37JOZH7ZI42Q, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.596607608 (594h30m17.467690644s) [noise=true , address=145.79.227.13:4804   , piece_id=VKWWH2L55TPWU3SFUGXPK3SIKYUPP4NOMFL4Z6O6QPIOHM45WREA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.702499053 (594h30m17.573582098s) [noise=true , address=77.90.31.14:4662     , piece_id=3CYNHYEFUEFWAY6SJA4ENNMNF22AKILNUW7N6DEP7X3ZUN7ZY6HQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.840169952 (594h30m17.711252988s) [noise=true , address=45.91.116.9:4263     , piece_id=IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.857328325 (594h30m17.72841136s) [noise=true , address=45.91.116.8:4047     , piece_id=V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.857350156 (594h30m17.728433191s) [seg_pos={0 0}, upload=521156]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:59:17.857707727 (594h30m17.728790762s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Dec 30 09:59:17.857714500 (594h30m17.728797535s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.857960509 (594h30m17.729043544s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection
Dec 30 09:59:17.857968544 (594h30m17.729051579s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.858124200 (594h30m17.729207235s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 30 09:59:17.858130543 (594h30m17.729213588s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.858165148 (594h30m17.729248184s) [seg_pos={0 0}, upload=521156]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:59:17.858267303 (594h30m17.729350338s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:59:17.858272162 (594h30m17.729355207s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:59:18.170209363 (594h30m18.041292398s) [upload=521156]: Done uploading segments...
Dec 30 09:59:18.170228529 (594h30m18.041311565s) [upload=521156]: Upload finished. err: <nil>
Duration: 5.969972646s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [....X....XXX..X.X.X.......XX....X..X.XX.........X..X......X.]
duration: 
3.570295786s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 10:00:09 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.570326105s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [....X........................................X..............]
duration: 
43.088389815s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 10:00:01 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 43.088422537s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [..X...X...X...X..............X....X.X..X.X..............XX.X]
duration: 
30.005467683s
trace: 
https://jaeger.dev.storj.io/trace/56455cb83c53e4bf
FAIL (2025-12-30 09:59:44 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, Piece ID: V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA) context deadline exceeded; piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA) context deadline exceeded
======LOGS======
30 09:59:15.268425076 (594h30m15.139508121s) [upload=521156]: Starting upload
Dec 30 09:59:15.268521350 (594h30m15.139604385s) [upload=521156]: Done starting upload
Dec 30 09:59:15.268677877 (594h30m15.139760922s) [upload=521156]: Uploading segments...
Dec 30 09:59:15.268960014 (594h30m15.140043049s) [upload=521156]: Got next segment. Inline: false
Dec 30 09:59:15.269045027 (594h30m15.140128072s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:59:15.269052902 (594h30m15.140135957s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:59:15.514504773 (594h30m15.385587808s) [seg_pos={0 0}, upload=521156]: Begin upload segment...
Dec 30 09:59:15.514555731 (594h30m15.385638766s) [seg_pos={0 0}, upload=521156]: Done begin upload segment.
Dec 30 09:59:15.514594785 (594h30m15.38567783s) [noise=true , address=145.79.227.13:4804   , piece_id=VKWWH2L55TPWU3SFUGXPK3SIKYUPP4NOMFL4Z6O6QPIOHM45WREA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.514613640 (594h30m15.385696686s) [noise=true , address=145.79.227.5:4221    , piece_id=BVGPFMIU2FKB3ODUCRFOL3MJGOJGCL2WGQM36YSNB55BVXPVYR5A, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515148249 (594h30m15.386231284s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515403585 (594h30m15.38648663s) [noise=true , address=145.79.227.36:4517   , piece_id=GJJK3NBZWQYDURWGZ3QMBIVYURJ2AIZYYZ3OJEDB6JRRSFU3DN6Q, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515704488 (594h30m15.386787523s) [noise=true , address=45.91.116.8:4047     , piece_id=V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.515994029 (594h30m15.387077064s) [noise=true , address=145.79.227.14:4840   , piece_id=OFKL6PZSWNLAI6S7RNHQE23S4RN5WPSPAQBXHMZR3MNOV2TYBW6A, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516246921 (594h30m15.387329966s) [noise=true , address=145.79.227.37:4566   , piece_id=BHKJQBDJHZ2NCPAR2Q37ZWVEF2XUDDLCCZZXRKHUCDQFY57HASBA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516610383 (594h30m15.387693418s) [noise=true , address=45.91.116.9:4263     , piece_id=IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.516924732 (594h30m15.388007767s) [noise=true , address=145.79.227.18:4006   , piece_id=QSSIOL3KRU5CACHELJ7QSXOW4CY6UX2PBS6AP5ENFFG4B257FQOQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.517179366 (594h30m15.388262401s) [noise=true , address=77.90.31.14:4662     , piece_id=3CYNHYEFUEFWAY6SJA4ENNMNF22AKILNUW7N6DEP7X3ZUN7ZY6HQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.517454601 (594h30m15.388537646s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518098957 (594h30m15.389181992s) [noise=true , address=145.79.227.41:4086   , piece_id=A6LE5QCA2YOC25K67X65IDZOGEH46BUUNQDEBXIE37JOZH7ZI42Q, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518550056 (594h30m15.389633101s) [noise=true , address=145.79.227.4:4187    , piece_id=KOV27O254S6MB6EJBFTW7MZHJJENSFKQFA5NFHOBELOBYLSK5PSA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.518925772 (594h30m15.390008807s) [noise=true , address=145.79.227.51:4410   , piece_id=CVJJR5M7CUXOY74ZCFC2BXLFUQORF57YXLWL6EIT7IAYP5VB5ZZA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.519337927 (594h30m15.390420962s) [noise=true , address=145.79.227.38:4978   , piece_id=J6G5HBREOULMW4X74BO6SJR6PXZ5SKXLBMUIB5IKTQP4F5VK24DQ, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:15.521755059 (594h30m15.392838104s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Uploading piece...
Dec 30 09:59:17.214727013 (594h30m17.085810048s) [upload=521156]: Next returned nil segment
Dec 30 09:59:17.214738596 (594h30m17.085821631s) [upload=521156]: Waiting for error group managing segments...
Dec 30 09:59:17.497607165 (594h30m17.368690201s) [noise=true , address=145.79.227.18:4006   , piece_id=QSSIOL3KRU5CACHELJ7QSXOW4CY6UX2PBS6AP5ENFFG4B257FQOQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.497722024 (594h30m17.368805059s) [noise=true , address=145.79.227.14:4840   , piece_id=OFKL6PZSWNLAI6S7RNHQE23S4RN5WPSPAQBXHMZR3MNOV2TYBW6A, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.501187864 (594h30m17.372270899s) [noise=true , address=145.79.227.36:4517   , piece_id=GJJK3NBZWQYDURWGZ3QMBIVYURJ2AIZYYZ3OJEDB6JRRSFU3DN6Q, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.503909276 (594h30m17.374992321s) [noise=true , address=145.79.227.5:4221    , piece_id=BVGPFMIU2FKB3ODUCRFOL3MJGOJGCL2WGQM36YSNB55BVXPVYR5A, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.511872969 (594h30m17.382956004s) [noise=true , address=145.79.227.51:4410   , piece_id=CVJJR5M7CUXOY74ZCFC2BXLFUQORF57YXLWL6EIT7IAYP5VB5ZZA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.515389095 (594h30m17.38647214s) [noise=true , address=145.79.227.38:4978   , piece_id=J6G5HBREOULMW4X74BO6SJR6PXZ5SKXLBMUIB5IKTQP4F5VK24DQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.539482172 (594h30m17.410565217s) [noise=true , address=145.79.227.4:4187    , piece_id=KOV27O254S6MB6EJBFTW7MZHJJENSFKQFA5NFHOBELOBYLSK5PSA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.584163543 (594h30m17.455246578s) [noise=true , address=145.79.227.37:4566   , piece_id=BHKJQBDJHZ2NCPAR2Q37ZWVEF2XUDDLCCZZXRKHUCDQFY57HASBA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.593999793 (594h30m17.465082828s) [noise=true , address=145.79.227.41:4086   , piece_id=A6LE5QCA2YOC25K67X65IDZOGEH46BUUNQDEBXIE37JOZH7ZI42Q, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.596607608 (594h30m17.467690644s) [noise=true , address=145.79.227.13:4804   , piece_id=VKWWH2L55TPWU3SFUGXPK3SIKYUPP4NOMFL4Z6O6QPIOHM45WREA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.702499053 (594h30m17.573582098s) [noise=true , address=77.90.31.14:4662     , piece_id=3CYNHYEFUEFWAY6SJA4ENNMNF22AKILNUW7N6DEP7X3ZUN7ZY6HQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.840169952 (594h30m17.711252988s) [noise=true , address=45.91.116.9:4263     , piece_id=IUUG3KM6ESVZH5J3CJ6WTOYG743SKFYUTZ7LMIDJT6WS5U5PCGZA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.857328325 (594h30m17.72841136s) [noise=true , address=45.91.116.8:4047     , piece_id=V4YRRIDX5E4GD6OFZ5RMRPZG7QFH6UYLETYXFKQEJYMCDCDXZGAA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: <nil>
Dec 30 09:59:17.857350156 (594h30m17.728433191s) [seg_pos={0 0}, upload=521156]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:59:17.857707727 (594h30m17.728790762s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Dec 30 09:59:17.857714500 (594h30m17.728797535s) [noise=true , address=145.79.227.34:4907   , piece_id=HPBCTYPTXRT5NKILA36DY6BGXCACD3CMPFGC3ILQ3YXKV73GFUVQ, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.857960509 (594h30m17.729043544s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57280->45.91.116.7:4009: use of closed network connection
Dec 30 09:59:17.857968544 (594h30m17.729051579s) [noise=true , address=45.91.116.7:4009     , piece_id=NFOVK5EUTT2EHX63EM5PYUPM2YRFK5ZHIDZKDE62IXVZS4BCONWA, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.858124200 (594h30m17.729207235s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 30 09:59:17.858130543 (594h30m17.729213588s) [noise=true , address=45.91.116.12:4228    , piece_id=JSXRWYOECNSKGTGL5A3TVMRWQR6OIHZQMDNVIV3S3VK5E6VRZYQQ, seg_pos={0 0}, upload=521156]: Long tail cancellation due to optimal threshold
Dec 30 09:59:17.858165148 (594h30m17.729248184s) [seg_pos={0 0}, upload=521156]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:59:17.858267303 (594h30m17.729350338s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:59:17.858272162 (594h30m17.729355207s) [upload=521156]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:59:18.170209363 (594h30m18.041292398s) [upload=521156]: Done uploading segments...
Dec 30 09:59:18.170228529 (594h30m18.041311565s) [upload=521156]: Upload finished. err: <nil>
Duration: 30.005494273s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [.....X....XXX..X.X.X.......XX....X..X.XX.........X..X......X]
duration: 
5.001847452s
trace: 
https://jaeger.dev.storj.io/trace/c9bea8911ae340
FAIL (2025-12-30 09:59:29 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001870336s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.....X........................................X.............]
duration: 
23.334091665s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:59:07 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 23.33410983s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X......]
duration: 
5.001436949s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:59:07 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001464241s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [......X........................................X............]
duration: 
37.657910947s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:58:43 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 37.657942135s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
11 out of 60 is failed (threshold=3) [...X...X...X...X..............X....X.X..X.X..............XX.]
duration: 
6.057312727s
trace: 
https://jaeger.dev.storj.io/trace/688915ffe92dfda9
FAIL (2025-12-30 09:58:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, Piece ID: TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ) context deadline exceeded
======LOGS======
 30 09:57:56.602581248 (594h28m56.473664283s) [upload=521136]: Starting upload
Dec 30 09:57:56.602658937 (594h28m56.473741982s) [upload=521136]: Done starting upload
Dec 30 09:57:56.602871431 (594h28m56.473954486s) [upload=521136]: Uploading segments...
Dec 30 09:57:56.603154720 (594h28m56.474237755s) [upload=521136]: Got next segment. Inline: false
Dec 30 09:57:56.603199866 (594h28m56.474282901s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:57:56.603204465 (594h28m56.4742875s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:57:56.848755638 (594h28m56.719838673s) [seg_pos={0 0}, upload=521136]: Begin upload segment...
Dec 30 09:57:56.848822595 (594h28m56.71990563s) [seg_pos={0 0}, upload=521136]: Done begin upload segment.
Dec 30 09:57:56.848857972 (594h28m56.719941018s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.848992839 (594h28m56.720075874s) [noise=true , address=145.79.227.36:4518   , piece_id=PTDEEE6PONYPBBY5FYP3XAIBKMS3TLE2UAQPYZTKSPHXTVDZGNSQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.849535954 (594h28m56.720618989s) [noise=true , address=145.79.227.51:4410   , piece_id=4TR5VAWR7IKOKU43ZIY7YXE5APWNL5ANAG3G6LSE7GNAFJICFQNA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850057617 (594h28m56.721140652s) [noise=true , address=145.79.227.9:4366    , piece_id=PW3IGHKM6QRITDCSK2VVZI3CKSVJRHGSZRWME5EX7442EAUJLZRA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850479471 (594h28m56.721562506s) [noise=true , address=77.90.31.11:4556     , piece_id=EXHUTM5JHOGGWSKU4FHRKTQVG6FOWWJ5PTH667TJ4Z7J54L5TW4Q, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850836129 (594h28m56.721919164s) [noise=true , address=45.91.116.7:4011     , piece_id=TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851181337 (594h28m56.722264372s) [noise=true , address=145.79.227.48:4301   , piece_id=GRBSIRZ4ZJVU6BHWW2QADQSFXGM4LZWJZSL5OS27XGXNIGHGIGFA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851513460 (594h28m56.722596495s) [noise=true , address=145.79.227.17:4971   , piece_id=XPEID4ZT5SFKOLNHGDEETOBAQG4KM4Q5F5IAERDMETYDIYZ42EBA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851763806 (594h28m56.722846841s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.853851462 (594h28m56.724934507s) [noise=true , address=145.79.227.50:4373   , piece_id=B5MEEPANXJC3SPF3OWY65TQJ4B4OBPOR6K5OJELNBYWEI2ALC6DA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854173755 (594h28m56.72525679s) [noise=true , address=45.91.116.5:4299     , piece_id=S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854458096 (594h28m56.725541131s) [noise=true , address=145.79.227.18:4007   , piece_id=Y73OY6FUFSS6PI2IUJZFWRZZSQIIY3KFLM6M4W6ULYXJCZPT4STQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854740254 (594h28m56.725823289s) [noise=true , address=145.79.227.35:4483   , piece_id=F5GZP7BZ7ODQIK5YH2NPGNQ4U5XJNRHNE5NDEGGZ46PX5XLMMVOQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855004347 (594h28m56.726087382s) [noise=true , address=145.79.227.3:4149    , piece_id=UDOSH2Q5THSQNAOJKAVQF6JPHW2G4BTAASNE7632XAENI7TP4D6A, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855299058 (594h28m56.726382113s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855607776 (594h28m56.726690821s) [noise=true , address=145.79.227.5:4222    , piece_id=IPULU7CZ7THELSF2T5K3SX3YOIEKPS6INMZCUBBVKQTPSFVV3T3Q, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:58.549826231 (594h28m58.420909266s) [upload=521136]: Next returned nil segment
Dec 30 09:57:58.549837813 (594h28m58.420920848s) [upload=521136]: Waiting for error group managing segments...
Dec 30 09:57:58.838137686 (594h28m58.709220721s) [noise=true , address=145.79.227.50:4373   , piece_id=B5MEEPANXJC3SPF3OWY65TQJ4B4OBPOR6K5OJELNBYWEI2ALC6DA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.838157764 (594h28m58.709240799s) [noise=true , address=145.79.227.18:4007   , piece_id=Y73OY6FUFSS6PI2IUJZFWRZZSQIIY3KFLM6M4W6ULYXJCZPT4STQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.839406742 (594h28m58.710489777s) [noise=true , address=145.79.227.17:4971   , piece_id=XPEID4ZT5SFKOLNHGDEETOBAQG4KM4Q5F5IAERDMETYDIYZ42EBA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.849035016 (594h28m58.720118051s) [noise=true , address=145.79.227.36:4518   , piece_id=PTDEEE6PONYPBBY5FYP3XAIBKMS3TLE2UAQPYZTKSPHXTVDZGNSQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.851529056 (594h28m58.722612091s) [noise=true , address=145.79.227.51:4410   , piece_id=4TR5VAWR7IKOKU43ZIY7YXE5APWNL5ANAG3G6LSE7GNAFJICFQNA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.856116560 (594h28m58.727199595s) [noise=true , address=145.79.227.3:4149    , piece_id=UDOSH2Q5THSQNAOJKAVQF6JPHW2G4BTAASNE7632XAENI7TP4D6A, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.856793931 (594h28m58.727876966s) [noise=true , address=145.79.227.9:4366    , piece_id=PW3IGHKM6QRITDCSK2VVZI3CKSVJRHGSZRWME5EX7442EAUJLZRA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.871439278 (594h28m58.742522313s) [noise=true , address=145.79.227.5:4222    , piece_id=IPULU7CZ7THELSF2T5K3SX3YOIEKPS6INMZCUBBVKQTPSFVV3T3Q, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.931160157 (594h28m58.802243202s) [noise=true , address=145.79.227.48:4301   , piece_id=GRBSIRZ4ZJVU6BHWW2QADQSFXGM4LZWJZSL5OS27XGXNIGHGIGFA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.370495916 (594h28m59.241578951s) [noise=true , address=45.91.116.7:4011     , piece_id=TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.397619894 (594h28m59.26870293s) [noise=true , address=45.91.116.5:4299     , piece_id=S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.425611315 (594h28m59.29669435s) [noise=true , address=145.79.227.35:4483   , piece_id=F5GZP7BZ7ODQIK5YH2NPGNQ4U5XJNRHNE5NDEGGZ46PX5XLMMVOQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.468231651 (594h28m59.339314696s) [noise=true , address=77.90.31.11:4556     , piece_id=EXHUTM5JHOGGWSKU4FHRKTQVG6FOWWJ5PTH667TJ4Z7J54L5TW4Q, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.468250107 (594h28m59.339333142s) [seg_pos={0 0}, upload=521136]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:57:59.468399912 (594h28m59.339482947s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:59.468405592 (594h28m59.339488647s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468643616 (594h28m59.339726661s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:59.468652833 (594h28m59.339735888s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468764607 (594h28m59.339847652s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection
Dec 30 09:57:59.468769957 (594h28m59.339852992s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468784875 (594h28m59.33986791s) [seg_pos={0 0}, upload=521136]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:57:59.468847274 (594h28m59.339930309s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:57:59.468852093 (594h28m59.339935128s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:57:59.752431445 (594h28m59.62351449s) [upload=521136]: Done uploading segments...
Dec 30 09:57:59.752454580 (594h28m59.623537615s) [upload=521136]: Upload finished. err: <nil>
Duration: 6.057341723s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [X...X...X...X...X..............X....X.X..X.X..............XX]
duration: 
30.003361261s
trace: 
https://jaeger.dev.storj.io/trace/688915ffe92dfda9
FAIL (2025-12-30 09:58:25 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, Piece ID: TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA) context deadline exceeded; piecestore: (Node ID: 12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, Piece ID: S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ) context deadline exceeded
======LOGS======
 30 09:57:56.602581248 (594h28m56.473664283s) [upload=521136]: Starting upload
Dec 30 09:57:56.602658937 (594h28m56.473741982s) [upload=521136]: Done starting upload
Dec 30 09:57:56.602871431 (594h28m56.473954486s) [upload=521136]: Uploading segments...
Dec 30 09:57:56.603154720 (594h28m56.474237755s) [upload=521136]: Got next segment. Inline: false
Dec 30 09:57:56.603199866 (594h28m56.474282901s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:57:56.603204465 (594h28m56.4742875s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:57:56.848755638 (594h28m56.719838673s) [seg_pos={0 0}, upload=521136]: Begin upload segment...
Dec 30 09:57:56.848822595 (594h28m56.71990563s) [seg_pos={0 0}, upload=521136]: Done begin upload segment.
Dec 30 09:57:56.848857972 (594h28m56.719941018s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.848992839 (594h28m56.720075874s) [noise=true , address=145.79.227.36:4518   , piece_id=PTDEEE6PONYPBBY5FYP3XAIBKMS3TLE2UAQPYZTKSPHXTVDZGNSQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.849535954 (594h28m56.720618989s) [noise=true , address=145.79.227.51:4410   , piece_id=4TR5VAWR7IKOKU43ZIY7YXE5APWNL5ANAG3G6LSE7GNAFJICFQNA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850057617 (594h28m56.721140652s) [noise=true , address=145.79.227.9:4366    , piece_id=PW3IGHKM6QRITDCSK2VVZI3CKSVJRHGSZRWME5EX7442EAUJLZRA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850479471 (594h28m56.721562506s) [noise=true , address=77.90.31.11:4556     , piece_id=EXHUTM5JHOGGWSKU4FHRKTQVG6FOWWJ5PTH667TJ4Z7J54L5TW4Q, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.850836129 (594h28m56.721919164s) [noise=true , address=45.91.116.7:4011     , piece_id=TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851181337 (594h28m56.722264372s) [noise=true , address=145.79.227.48:4301   , piece_id=GRBSIRZ4ZJVU6BHWW2QADQSFXGM4LZWJZSL5OS27XGXNIGHGIGFA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851513460 (594h28m56.722596495s) [noise=true , address=145.79.227.17:4971   , piece_id=XPEID4ZT5SFKOLNHGDEETOBAQG4KM4Q5F5IAERDMETYDIYZ42EBA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.851763806 (594h28m56.722846841s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.853851462 (594h28m56.724934507s) [noise=true , address=145.79.227.50:4373   , piece_id=B5MEEPANXJC3SPF3OWY65TQJ4B4OBPOR6K5OJELNBYWEI2ALC6DA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854173755 (594h28m56.72525679s) [noise=true , address=45.91.116.5:4299     , piece_id=S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854458096 (594h28m56.725541131s) [noise=true , address=145.79.227.18:4007   , piece_id=Y73OY6FUFSS6PI2IUJZFWRZZSQIIY3KFLM6M4W6ULYXJCZPT4STQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.854740254 (594h28m56.725823289s) [noise=true , address=145.79.227.35:4483   , piece_id=F5GZP7BZ7ODQIK5YH2NPGNQ4U5XJNRHNE5NDEGGZ46PX5XLMMVOQ, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855004347 (594h28m56.726087382s) [noise=true , address=145.79.227.3:4149    , piece_id=UDOSH2Q5THSQNAOJKAVQF6JPHW2G4BTAASNE7632XAENI7TP4D6A, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855299058 (594h28m56.726382113s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:56.855607776 (594h28m56.726690821s) [noise=true , address=145.79.227.5:4222    , piece_id=IPULU7CZ7THELSF2T5K3SX3YOIEKPS6INMZCUBBVKQTPSFVV3T3Q, seg_pos={0 0}, upload=521136]: Uploading piece...
Dec 30 09:57:58.549826231 (594h28m58.420909266s) [upload=521136]: Next returned nil segment
Dec 30 09:57:58.549837813 (594h28m58.420920848s) [upload=521136]: Waiting for error group managing segments...
Dec 30 09:57:58.838137686 (594h28m58.709220721s) [noise=true , address=145.79.227.50:4373   , piece_id=B5MEEPANXJC3SPF3OWY65TQJ4B4OBPOR6K5OJELNBYWEI2ALC6DA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.838157764 (594h28m58.709240799s) [noise=true , address=145.79.227.18:4007   , piece_id=Y73OY6FUFSS6PI2IUJZFWRZZSQIIY3KFLM6M4W6ULYXJCZPT4STQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.839406742 (594h28m58.710489777s) [noise=true , address=145.79.227.17:4971   , piece_id=XPEID4ZT5SFKOLNHGDEETOBAQG4KM4Q5F5IAERDMETYDIYZ42EBA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.849035016 (594h28m58.720118051s) [noise=true , address=145.79.227.36:4518   , piece_id=PTDEEE6PONYPBBY5FYP3XAIBKMS3TLE2UAQPYZTKSPHXTVDZGNSQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.851529056 (594h28m58.722612091s) [noise=true , address=145.79.227.51:4410   , piece_id=4TR5VAWR7IKOKU43ZIY7YXE5APWNL5ANAG3G6LSE7GNAFJICFQNA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.856116560 (594h28m58.727199595s) [noise=true , address=145.79.227.3:4149    , piece_id=UDOSH2Q5THSQNAOJKAVQF6JPHW2G4BTAASNE7632XAENI7TP4D6A, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.856793931 (594h28m58.727876966s) [noise=true , address=145.79.227.9:4366    , piece_id=PW3IGHKM6QRITDCSK2VVZI3CKSVJRHGSZRWME5EX7442EAUJLZRA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.871439278 (594h28m58.742522313s) [noise=true , address=145.79.227.5:4222    , piece_id=IPULU7CZ7THELSF2T5K3SX3YOIEKPS6INMZCUBBVKQTPSFVV3T3Q, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:58.931160157 (594h28m58.802243202s) [noise=true , address=145.79.227.48:4301   , piece_id=GRBSIRZ4ZJVU6BHWW2QADQSFXGM4LZWJZSL5OS27XGXNIGHGIGFA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.370495916 (594h28m59.241578951s) [noise=true , address=45.91.116.7:4011     , piece_id=TZUHQCLJ4IYJQF7PV7IAMJMIG2JV34CP7O24AP2K252GAEQZLJNA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.397619894 (594h28m59.26870293s) [noise=true , address=45.91.116.5:4299     , piece_id=S2F5SOWGSWPBGTDC22T6DROE4DTO3GINGUJ6LGZGKXRIACR6GWXQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.425611315 (594h28m59.29669435s) [noise=true , address=145.79.227.35:4483   , piece_id=F5GZP7BZ7ODQIK5YH2NPGNQ4U5XJNRHNE5NDEGGZ46PX5XLMMVOQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.468231651 (594h28m59.339314696s) [noise=true , address=77.90.31.11:4556     , piece_id=EXHUTM5JHOGGWSKU4FHRKTQVG6FOWWJ5PTH667TJ4Z7J54L5TW4Q, seg_pos={0 0}, upload=521136]: Done uploading piece. err: <nil>
Dec 30 09:57:59.468250107 (594h28m59.339333142s) [seg_pos={0 0}, upload=521136]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:57:59.468399912 (594h28m59.339482947s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:59.468405592 (594h28m59.339488647s) [noise=true , address=145.79.227.34:4906   , piece_id=J3Q34G6NYWTYZ37KKSLNEEYD5JJ2G77RLGVHK53MP47AEWNCVHFA, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468643616 (594h28m59.339726661s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:59.468652833 (594h28m59.339735888s) [noise=true , address=45.91.116.9:4261     , piece_id=RHPNOCIZOKGV774O252K7F3IWQGWFF3CX5ZT6BOKQ45XAWFFEEMA, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468764607 (594h28m59.339847652s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47666->45.91.116.12:4227: use of closed network connection
Dec 30 09:57:59.468769957 (594h28m59.339852992s) [noise=true , address=45.91.116.12:4227    , piece_id=KVQLZDNPEWATAXCNSITYEW2KXLJ6H5LXOV7GTWH2QR2Z2YFYJ4GQ, seg_pos={0 0}, upload=521136]: Long tail cancellation due to optimal threshold
Dec 30 09:57:59.468784875 (594h28m59.33986791s) [seg_pos={0 0}, upload=521136]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:57:59.468847274 (594h28m59.339930309s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:57:59.468852093 (594h28m59.339935128s) [upload=521136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:57:59.752431445 (594h28m59.62351449s) [upload=521136]: Done uploading segments...
Dec 30 09:57:59.752454580 (594h28m59.623537615s) [upload=521136]: Upload finished. err: <nil>
Duration: 30.003391519s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.......X........................................X...........]
duration: 
47.170686156s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:58:05 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 47.170712697s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [.X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X.....]
duration: 
3.58711109s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:57:52 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.587140296s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
11 out of 60 is failed (threshold=3) [.X...X...X...X...X..............X....X.X..X.X..............X]
duration: 
30.002348024s
trace: 
https://jaeger.dev.storj.io/trace/6079c8976e7f9f37
FAIL (2025-12-30 09:57:50 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 30 09:57:20.743015242 (594h28m20.614098287s) [upload=521127]: Starting upload
Dec 30 09:57:20.743120363 (594h28m20.614203408s) [upload=521127]: Done starting upload
Dec 30 09:57:20.743380609 (594h28m20.614463654s) [upload=521127]: Uploading segments...
Dec 30 09:57:20.743453056 (594h28m20.614536092s) [upload=521127]: Got next segment. Inline: false
Dec 30 09:57:20.743484857 (594h28m20.614567892s) [upload=521127]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:57:20.743488935 (594h28m20.61457197s) [upload=521127]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:57:20.994395626 (594h28m20.865478671s) [seg_pos={0 0}, upload=521127]: Begin upload segment...
Dec 30 09:57:20.994447906 (594h28m20.865530951s) [seg_pos={0 0}, upload=521127]: Done begin upload segment.
Dec 30 09:57:20.994479716 (594h28m20.865562761s) [noise=true , address=77.90.31.11:4554     , piece_id=3S7EOTYUNMHOPCMQ2TYXY3ZSVDZRDAJI47UR7IG7ARWL3C62V3RQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.994629962 (594h28m20.865713007s) [noise=true , address=145.79.227.8:4330    , piece_id=62FPTX2M2AIWTZ66YMUU7MI5BDN7RQYQC7EIA7DBXXW5LQZP6UUQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.995542960 (594h28m20.866625996s) [noise=true , address=145.79.227.36:4517   , piece_id=6R3C7UCS5OB5TL3RM4NFIMCYRKPOZCG6CLNTKIMXBDQBZF5QKOVQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.996172439 (594h28m20.867255474s) [noise=true , address=145.79.227.14:4838   , piece_id=RGVZFHX6WGH7RA3RIIRFGIUHSHE4I3THW3W4OAVB4DSJ2NLNSYRQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.996696447 (594h28m20.867779482s) [noise=true , address=145.79.227.44:4193   , piece_id=ZVAP442WDPCSBHZ3NMTHZVTIT7NQ2MS2HBWXXNPMSHO23MWFK3NA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.996854017 (594h28m20.867937062s) [noise=true , address=145.79.227.25:4619   , piece_id=BS7AATHTPN7VJLFR3EZ7ZEDCQLZGRYFLHLNOOA3ZSAGMHATJE6BA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.997898666 (594h28m20.868981712s) [noise=true , address=145.79.227.4:4186    , piece_id=FWIBFACV5WFXNWDXLO6IH7J4N7WYAWMUHHSE4JPZ6UU4JD2IER7Q, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.997938793 (594h28m20.869021838s) [noise=true , address=45.91.116.13:4153    , piece_id=2TTB7HWGIIEW5DYLSMPYBZRUCPEWX5L4ITCGJ7HJKOCCW7VMKSYA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.998691396 (594h28m20.869774441s) [noise=true , address=145.79.227.10:4402   , piece_id=QETWI6P4KIFKZFGKC3X7FB5VSUURVIGHIXVF2QVUFZGUJRH5L3RA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.999059246 (594h28m20.870142281s) [noise=true , address=77.90.31.21:4878     , piece_id=CCAZDDZDZVS6NF5SM434H67N3XOIAMFGIRJ3JB3MIPGZJ4IB326Q, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.999476761 (594h28m20.870559876s) [noise=true , address=45.91.116.7:4010     , piece_id=BAIXYEYXHJKLMFL5IZWXCJY6FMNOWMZUOYXEQL6CTPMSOEZ3DCLQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.999555561 (594h28m20.870638606s) [noise=true , address=145.79.227.3:4151    , piece_id=ETEG3WQUVGDBS46LFVS35IIVUAMAWVIENMKPWA4N2I3M4ONN4YSA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:20.999976032 (594h28m20.871059067s) [noise=true , address=145.79.227.48:4302   , piece_id=D6WNLZXABHESJ425M4MDGE5ISSJGX7SXAOINMEH2JV5E3VAMXZBQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:21.000079610 (594h28m20.871162645s) [noise=true , address=145.79.227.45:4229   , piece_id=3LR6ZY5DAUJUHGQT62HOSUUTFSODTSLZTBT5V6DA7HNZHEYQ5EMQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:21.000392656 (594h28m20.871475711s) [noise=true , address=145.79.227.9:4365    , piece_id=GMSSMJMAWQX6J2PUIZPKBYY3O4G3N4BI6TELLB4PGMLAPEY5KVEA, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:21.000649364 (594h28m20.871732409s) [noise=true , address=45.91.116.11:4120    , piece_id=MLM4HCLTH6OHZHWK2LFI5AJKEI7RBGVXXH47S5NH2N6W7T3YYVJQ, seg_pos={0 0}, upload=521127]: Uploading piece...
Dec 30 09:57:22.707111484 (594h28m22.578194529s) [upload=521127]: Next returned nil segment
Dec 30 09:57:22.707173802 (594h28m22.578256837s) [upload=521127]: Waiting for error group managing segments...
Dec 30 09:57:22.990460311 (594h28m22.861543346s) [noise=true , address=145.79.227.45:4229   , piece_id=3LR6ZY5DAUJUHGQT62HOSUUTFSODTSLZTBT5V6DA7HNZHEYQ5EMQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:22.997066901 (594h28m22.868149936s) [noise=true , address=145.79.227.9:4365    , piece_id=GMSSMJMAWQX6J2PUIZPKBYY3O4G3N4BI6TELLB4PGMLAPEY5KVEA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.007066742 (594h28m22.878149787s) [noise=true , address=145.79.227.8:4330    , piece_id=62FPTX2M2AIWTZ66YMUU7MI5BDN7RQYQC7EIA7DBXXW5LQZP6UUQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.012651357 (594h28m22.883734402s) [noise=true , address=145.79.227.36:4517   , piece_id=6R3C7UCS5OB5TL3RM4NFIMCYRKPOZCG6CLNTKIMXBDQBZF5QKOVQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.013286837 (594h28m22.884369872s) [noise=true , address=145.79.227.10:4402   , piece_id=QETWI6P4KIFKZFGKC3X7FB5VSUURVIGHIXVF2QVUFZGUJRH5L3RA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.013715753 (594h28m22.884798788s) [noise=true , address=145.79.227.25:4619   , piece_id=BS7AATHTPN7VJLFR3EZ7ZEDCQLZGRYFLHLNOOA3ZSAGMHATJE6BA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.018188601 (594h28m22.889271636s) [noise=true , address=145.79.227.4:4186    , piece_id=FWIBFACV5WFXNWDXLO6IH7J4N7WYAWMUHHSE4JPZ6UU4JD2IER7Q, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.018643277 (594h28m22.889726312s) [noise=true , address=145.79.227.14:4838   , piece_id=RGVZFHX6WGH7RA3RIIRFGIUHSHE4I3THW3W4OAVB4DSJ2NLNSYRQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.020153722 (594h28m22.891236757s) [noise=true , address=145.79.227.3:4151    , piece_id=ETEG3WQUVGDBS46LFVS35IIVUAMAWVIENMKPWA4N2I3M4ONN4YSA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.086737100 (594h28m22.957820145s) [noise=true , address=145.79.227.48:4302   , piece_id=D6WNLZXABHESJ425M4MDGE5ISSJGX7SXAOINMEH2JV5E3VAMXZBQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.289366381 (594h28m23.160449416s) [noise=true , address=77.90.31.11:4554     , piece_id=3S7EOTYUNMHOPCMQ2TYXY3ZSVDZRDAJI47UR7IG7ARWL3C62V3RQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.469500990 (594h28m23.340584025s) [noise=true , address=45.91.116.11:4120    , piece_id=MLM4HCLTH6OHZHWK2LFI5AJKEI7RBGVXXH47S5NH2N6W7T3YYVJQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.608765234 (594h28m23.479848279s) [noise=true , address=45.91.116.13:4153    , piece_id=2TTB7HWGIIEW5DYLSMPYBZRUCPEWX5L4ITCGJ7HJKOCCW7VMKSYA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: <nil>
Dec 30 09:57:23.608793909 (594h28m23.479876944s) [seg_pos={0 0}, upload=521127]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:57:23.608984081 (594h28m23.480067116s) [noise=true , address=77.90.31.21:4878     , piece_id=CCAZDDZDZVS6NF5SM434H67N3XOIAMFGIRJ3JB3MIPGZJ4IB326Q, seg_pos={0 0}, upload=521127]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:23.608990995 (594h28m23.48007403s) [noise=true , address=77.90.31.21:4878     , piece_id=CCAZDDZDZVS6NF5SM434H67N3XOIAMFGIRJ3JB3MIPGZJ4IB326Q, seg_pos={0 0}, upload=521127]: Long tail cancellation due to optimal threshold
Dec 30 09:57:23.609122254 (594h28m23.480205289s) [noise=true , address=45.91.116.7:4010     , piece_id=BAIXYEYXHJKLMFL5IZWXCJY6FMNOWMZUOYXEQL6CTPMSOEZ3DCLQ, seg_pos={0 0}, upload=521127]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:23.609132113 (594h28m23.480215158s) [noise=true , address=45.91.116.7:4010     , piece_id=BAIXYEYXHJKLMFL5IZWXCJY6FMNOWMZUOYXEQL6CTPMSOEZ3DCLQ, seg_pos={0 0}, upload=521127]: Long tail cancellation due to optimal threshold
Dec 30 09:57:23.609208659 (594h28m23.480291704s) [noise=true , address=145.79.227.44:4193   , piece_id=ZVAP442WDPCSBHZ3NMTHZVTIT7NQ2MS2HBWXXNPMSHO23MWFK3NA, seg_pos={0 0}, upload=521127]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF
Dec 30 09:57:23.609215001 (594h28m23.480298036s) [noise=true , address=145.79.227.44:4193   , piece_id=ZVAP442WDPCSBHZ3NMTHZVTIT7NQ2MS2HBWXXNPMSHO23MWFK3NA, seg_pos={0 0}, upload=521127]: Long tail cancellation due to optimal threshold
Dec 30 09:57:23.609237594 (594h28m23.480320639s) [seg_pos={0 0}, upload=521127]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:57:23.609301396 (594h28m23.480384431s) [upload=521127]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:57:23.609306125 (594h28m23.48038916s) [upload=521127]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:57:23.906231964 (594h28m23.777314999s) [upload=521127]: Done uploading segments...
Dec 30 09:57:23.906251711 (594h28m23.777334756s) [upload=521127]: Upload finished. err: <nil>
Duration: 30.002364036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [........X........................................X..........]
duration: 
33.132213332s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:57:10 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 33.132229593s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [..X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X....]
duration: 
3.8828851s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:56:51 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.882913603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.........X........................................X.........]
duration: 
37.392899545s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:56:37 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 37.392930153s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [..X...X...X...X...X..............X....X.X..X.X..............]
duration: 
6.149329068s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:56:27 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 6.149363172s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [...X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X...]
duration: 
3.475934795s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:56:22 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.475962388s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [....X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X..]
duration: 
3.395465283s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:55:51 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.39549528s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..........X........................................X........]
duration: 
37.756939631s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:55:31 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 37.756956495s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [...X...X...X...X...X..............X....X.X..X.X.............]
duration: 
5.779702429s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:55:15 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 5.779738487s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [.....X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X.]
duration: 
3.37938741s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:55:03 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.379418899s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...........X........................................X.......]
duration: 
30.379737815s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:54:53 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 30.379763464s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [....X...X...X...X...X..............X....X.X..X.X............]
duration: 
5.759573631s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:54:40 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 5.759602505s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
16 out of 60 is failed (threshold=6) [......X.....X....XXX..X.X.X.......XX....X..X.XX.........X..X]
duration: 
5.001417195s
trace: 
https://jaeger.dev.storj.io/trace/1eb89afb78b1598
FAIL (2025-12-30 09:54:27 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00144141s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [.....X...X...X...X...X..............X....X.X..X.X...........]
duration: 
5.676721824s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:54:01 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 5.676752011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [............X........................................X......]
duration: 
38.123863609s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:53:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 38.123876974s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [.......X.....X....XXX..X.X.X.......XX....X..X.XX.........X..]
duration: 
4.888262278s
trace: 
https://jaeger.dev.storj.io/trace/4f252342701c86b1
FAIL (2025-12-30 09:53:38 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.888290641s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [......X...X...X...X...X..............X....X.X..X.X..........]
duration: 
6.140213877s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:53:24 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 6.140247881s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.............X........................................X.....]
duration: 
51.461432403s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:53:18 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 51.461461319s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
11 out of 60 is failed (threshold=3) [X......X...X...X...X...X..............X....X.X..X.X.........]
duration: 
5.858606646s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:53:02 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 5.858638066s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [XX......X...X...X...X...X..............X....X.X..X.X........]
duration: 
5.948084748s
trace: 
https://jaeger.dev.storj.io/trace/6f14fe4de74eb513
FAIL (2025-12-30 09:52:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, Piece ID: AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA) context deadline exceeded; piecestore: (Node ID: 12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, Piece ID: P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ) context deadline exceeded; piecestore: (Node ID: 1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, Piece ID: FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA) context deadline exceeded
======LOGS======
Dec 30 09:45:54.941531524 (594h16m54.812614559s) [upload=520962]: Starting upload
Dec 30 09:45:54.941589874 (594h16m54.812672919s) [upload=520962]: Done starting upload
Dec 30 09:45:54.941696047 (594h16m54.812779082s) [upload=520962]: Uploading segments...
Dec 30 09:45:54.941830342 (594h16m54.812913377s) [upload=520962]: Got next segment. Inline: false
Dec 30 09:45:54.941864998 (594h16m54.812948033s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:45:54.941870439 (594h16m54.812953484s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:45:55.188150120 (594h16m55.059233155s) [seg_pos={0 0}, upload=520962]: Begin upload segment...
Dec 30 09:45:55.188208361 (594h16m55.059291396s) [seg_pos={0 0}, upload=520962]: Done begin upload segment.
Dec 30 09:45:55.188236093 (594h16m55.059319128s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.190766401 (594h16m55.061849437s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191205067 (594h16m55.062288102s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191559843 (594h16m55.062642878s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.191886033 (594h16m55.062969068s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192177659 (594h16m55.063260694s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192531202 (594h16m55.063614237s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.192885697 (594h16m55.063968732s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193273525 (594h16m55.064356571s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.193648560 (594h16m55.064731595s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195007888 (594h16m55.066090923s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.195769709 (594h16m55.066852754s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196247128 (594h16m55.067330163s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.196701584 (594h16m55.067784619s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197200343 (594h16m55.068283378s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:55.197560909 (594h16m55.068643944s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Uploading piece...
Dec 30 09:45:56.800840511 (594h16m56.671923556s) [upload=520962]: Next returned nil segment
Dec 30 09:45:56.800859266 (594h16m56.671942312s) [upload=520962]: Waiting for error group managing segments...
Dec 30 09:45:57.180793620 (594h16m57.051876655s) [noise=true , address=145.79.227.8:4331    , piece_id=NPJZ2PDKQ75IJSLEEVLCXXBZT4YHJC5HWUQ4WHGB3PYLQ6Y6KYFA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.180977029 (594h16m57.052060064s) [noise=true , address=145.79.227.13:4804   , piece_id=AE3J4JDF63QOTC7PBIBC3PH2XUGQSQJFOPBI2QNVU7TBRUY2EDXQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.182409035 (594h16m57.05349208s) [noise=true , address=145.79.227.23:4545   , piece_id=LWN2PS56VYYJA5XWNFJNJLJYNVX5ANZODKXO4DQOSXFLCM47NEKA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184429493 (594h16m57.055512528s) [noise=true , address=145.79.227.38:4977   , piece_id=5MHEJZ2XADRIZUWA34FCQEPDZGYSO6PIAAQLYC4IQ5YUNK7OPMOA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.184776303 (594h16m57.055859348s) [noise=true , address=145.79.227.26:4654   , piece_id=FUQFOTOIAVOPINQIY3W4CPY72M4QV4ZCX265V5E74ZRWKR7RSHEA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.189334102 (594h16m57.060417137s) [noise=true , address=145.79.227.42:4122   , piece_id=BBNLWPEKYIZ7IYB5DS7J3HU2VLZTFQGBV5NK7EBKQPIS4WWV4A5Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.205424692 (594h16m57.076507727s) [noise=true , address=145.79.227.31:4799   , piece_id=4YDYJQUBL6H45E2YE5BUXY433N5ZRZ4AKYSP5UPIC6XZB2C6UF2Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.211881306 (594h16m57.082964341s) [noise=true , address=145.79.227.4:4185    , piece_id=TSGAHRL2O6MWKQ43HEODKEZ754U7EZ4Y4QAEMT3FSYF74NDAMV4Q, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216035707 (594h16m57.087118742s) [noise=true , address=145.79.227.11:4732   , piece_id=M2TNPOXLE4T25TE3OM3RYBWU4DBUIZGSGJUUNW42BVKXLKZBNTKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.216935862 (594h16m57.088018907s) [noise=true , address=145.79.227.7:4293    , piece_id=OIEJPGKYT4VA24L3HF4U3NZBCZHGOGDHV2UORLBJSMLUKYKCN3CQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.489351249 (594h16m57.360434284s) [noise=true , address=45.91.116.7:4009     , piece_id=4JYAJMPWOKFLVBPTJ64UVGMK7AO77HKKSQB4TEPNPVOECLEAPFRQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.620587466 (594h16m57.491670511s) [noise=true , address=45.91.116.12:4227    , piece_id=P6YSNR5NSBDXSCHBOIWDQYB3OEMXXJY5ASPKLHDD3OIYMFC3YMEQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777689968 (594h16m57.648773013s) [noise=true , address=45.91.116.5:4297     , piece_id=AAOFIGDMNL5REJVCEWUUHGWFB3IQGGD2GX5TZXAYZIS2BALRJISA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: <nil>
Dec 30 09:45:57.777709063 (594h16m57.648792098s) [seg_pos={0 0}, upload=520962]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:45:57.777894647 (594h16m57.648977682s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.777901220 (594h16m57.648984255s) [noise=true , address=145.79.227.35:4481   , piece_id=OBFCWZ6LRTPORWUFT6LL4XDSNF4MOJYVWL4B43BNAO5VYOY7CSKQ, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778007763 (594h16m57.649090798s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778012682 (594h16m57.649095727s) [noise=true , address=45.91.116.8:4045     , piece_id=G5UM3TZBIAVSA57TSVYQKZJCOPGVFAIDJQ3TRJYWD2JM6OVEN5CA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778109687 (594h16m57.649192732s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Dec 30 09:45:57.778115468 (594h16m57.649198503s) [noise=true , address=45.91.116.6:4975     , piece_id=F4LKZ5AACJHXBHNQUX2MD3B64DEE4MMZDN44RXAHUJESFVQXRFNA, seg_pos={0 0}, upload=520962]: Long tail cancellation due to optimal threshold
Dec 30 09:45:57.778129955 (594h16m57.64921299s) [seg_pos={0 0}, upload=520962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:45:57.778199066 (594h16m57.649282101s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:45:57.778203846 (594h16m57.649286881s) [upload=520962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:45:58.065674462 (594h16m57.936757517s) [upload=520962]: Done uploading segments...
Dec 30 09:45:58.065696523 (594h16m57.936779568s) [upload=520962]: Upload finished. err: <nil>
Duration: 5.948108553s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
15 out of 60 is failed (threshold=6) [........X.....X....XXX..X.X.X.......XX....X..X.XX.........X.]
duration: 
3.891448465s
trace: 
https://jaeger.dev.storj.io/trace/4f252342701c86b1
FAIL (2025-12-30 09:52:26 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.891466389s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..............X........................................X....]
duration: 
15.46012519s
trace: 
https://jaeger.dev.storj.io/trace/3195355f13f05d36
FAIL (2025-12-30 09:52:21 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067019971 (594h19m17.938103006s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.067303912 (594h19m17.938386947s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068459432 (594h19m17.939542467s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.068997457 (594h19m17.940080502s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.069543326 (594h19m17.940626371s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070650734 (594h19m17.941733779s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.070875393 (594h19m17.941958428s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.071784003 (594h19m17.942867048s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.073711273 (594h19m17.944794318s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.075235234 (594h19m17.946318279s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.076829440 (594h19m17.947912515s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078272018 (594h19m17.949355063s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.078661900 (594h19m17.949744935s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080047989 (594h19m17.951131024s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:18.080417162 (594h19m17.951500207s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Uploading piece...
Dec 30 09:48:19.524439219 (594h19m19.395522254s) [upload=520994]: Next returned nil segment
Dec 30 09:48:19.524450460 (594h19m19.395533495s) [upload=520994]: Waiting for error group managing segments...
Dec 30 09:48:19.918997407 (594h19m19.790080443s) [noise=true , address=145.79.227.13:4824   , piece_id=M3SJB242OYQKPPNRZQRYBJUY5IIW57IBIRNUYTL7ZGP72G4PDQYQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.996316117 (594h19m19.867399162s) [noise=true , address=64.49.13.26:4946     , piece_id=VKIRJBY4ERVNZLWECAXQ5XBZSHQXNWLICXL2SH7L6YJTAOMQ4B5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:19.998204242 (594h19m19.869287287s) [noise=true , address=145.79.227.29:4951   , piece_id=KAF5HJKOCR5CWF5NLYHF4AEMOHCXKCF5IB4AYSMKX5TVSJ56ATFA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.091699153 (594h19m19.962782198s) [noise=true , address=77.90.31.14:4674     , piece_id=6GZYDZAGRV5EE7C4ABVQI3FJHMNIMPKPN7MHF3W5CAG7YSDECU6Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.094776833 (594h19m19.965859868s) [noise=true , address=77.90.31.21:4901     , piece_id=GND6INAI7OCC3F5YVVBDBUQ4MKP6NUNOP3GTNKYPRDKHDLZ3HB7Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.125557174 (594h19m19.996640209s) [noise=true , address=77.90.31.18:4821     , piece_id=JJ2MBP4YE4ASKT34AS43HJH76O6ZS25QQM7E6JPDPYQ2HGVRXNRA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.190831969 (594h19m20.061915015s) [noise=true , address=77.90.31.19:4849     , piece_id=CPULLHYLFYY62ZMJWHDXF5TAXTGJYDXWZF4GQEHZBZTNSTFSNPTQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.202189506 (594h19m20.073272551s) [noise=true , address=77.90.31.17:4779     , piece_id=GLSAXG7T6S3JTFVS7HQDISHG3UDV7TUIYO5L2JOQSAVWOQYUQHQQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.256867624 (594h19m20.127950659s) [noise=true , address=77.90.31.10:4538     , piece_id=35EZE43JQLU3QS2V4ADSBVVSKY4ADW7L3HMZ6ATDKQVBM3LFDH5A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.287664355 (594h19m20.15874739s) [noise=true , address=77.90.31.12:4608     , piece_id=JQ74XBBKJFVURSNSLTAFCOI5O254QZWMB3NFQ7WYJP6OJ2OWMSUA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.827809488 (594h19m20.698892523s) [noise=true , address=145.79.227.30:4765   , piece_id=UZK5TI3566QAEMMHWFRV2IEPZFAZXJTPCQ7YRS5KZVUVXFTTQNEQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.877420858 (594h19m20.748503903s) [noise=true , address=145.79.227.44:4213   , piece_id=YSD4J4VYC3YWB5GSSGY6LWELBAXX62VVWFEGDT7ROFV6MWYEDRRQ, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:20.990976158 (594h19m20.862059203s) [noise=true , address=193.149.164.143:4699 , piece_id=P5SMN6VOYPZXI2YMBY6EMPUAGFE3ABPEJ3NSYYFYGID4MD4PDNQA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006244470 (594h19m20.877327506s) [noise=true , address=145.79.227.33:4877   , piece_id=QLY5EMHVDQYWKF4EDJTVDC2WI66VMY6EUU6BMDS4KB6R4ZEZT62Q, seg_pos={0 0}, upload=520994]: Done uploading piece. err: <nil>
Dec 30 09:48:21.006262295 (594h19m20.87734533s) [seg_pos={0 0}, upload=520994]: Segment reached optimal threshold of 14 pieces.
Dec 30 09:48:21.006858620 (594h19m20.877941655s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011274880 (594h19m20.882357925s) [noise=true , address=145.79.227.40:4053   , piece_id=JNLIJDJ6UVS3SXJMOH4QRGA2VA6B2AVKRMIB44NOORBMAE6GMUWA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011309486 (594h19m20.882392531s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12B6ymfqNvqiDsAbkF8H4ExxEuYaYb7gYQ45Y5GF4PGAAwmjbyd): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011320236 (594h19m20.882403271s) [noise=true , address=145.79.227.34:4928   , piece_id=EIMXDAKABH73MGYGDOLEZ3NPMJ6DTIQMVENWBGZ4WWTCRMWEQZZA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011340335 (594h19m20.88242338s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ArtdfDCemaBzgtSYdVnuvLYUmKc4pgGLQEjmP1K1Wn4eoKTp4): protocol: expected piece hash; context canceled; EOF
Dec 30 09:48:21.011349221 (594h19m20.882432266s) [noise=true , address=45.91.116.5:4308     , piece_id=JKYGMAAW5KR7MRDJI4HXRXO4IM3A6LH6TFYA5LNRSTRCQ3S6AG3A, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011359190 (594h19m20.882442225s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42790->64.49.13.169:4715: use of closed network connection
Dec 30 09:48:21.011406901 (594h19m20.882489946s) [noise=true , address=64.49.13.169:4715    , piece_id=Y6TATIERO6IKFUFSX3XXIQ2BRDEROCNHNQJRYRJIXOG6W4QWZLVA, seg_pos={0 0}, upload=520994]: Long tail cancellation due to optimal threshold
Dec 30 09:48:21.011481854 (594h19m20.882564899s) [seg_pos={0 0}, upload=520994]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 09:48:21.011589038 (594h19m20.882672083s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:48:21.011596151 (594h19m20.882679196s) [upload=520994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:48:21.309708250 (594h19m21.180791285s) [upload=520994]: Done uploading segments...
Dec 30 09:48:21.309729050 (594h19m21.180812085s) [upload=520994]: Upload finished. err: <nil>
Duration: 15.460149806s core uplink us-select-1 us1-placement-20