Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 902995 reports (Page 11301 of 18060)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [......X...X............XX....X.X..X...X..X...X...X...X.....X]
duration: 
30.004505139s
trace: 
https://jaeger.dev.storj.io/trace/7e20234e78421fd4
FAIL (2025-12-29 08:30:10 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 29 08:29:41.451512372 (569h0m41.322595417s) [upload=498874]: Starting upload
Dec 29 08:29:41.451575412 (569h0m41.322658447s) [upload=498874]: Done starting upload
Dec 29 08:29:41.451677667 (569h0m41.322760712s) [upload=498874]: Uploading segments...
Dec 29 08:29:41.451966006 (569h0m41.323049051s) [upload=498874]: Got next segment. Inline: false
Dec 29 08:29:41.451995652 (569h0m41.323078687s) [upload=498874]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:29:41.452000632 (569h0m41.323083677s) [upload=498874]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:29:41.697094378 (569h0m41.568177433s) [seg_pos={0 0}, upload=498874]: Begin upload segment...
Dec 29 08:29:41.697146256 (569h0m41.568229301s) [seg_pos={0 0}, upload=498874]: Done begin upload segment.
Dec 29 08:29:41.697187284 (569h0m41.568270329s) [noise=true , address=77.90.31.17:4773     , piece_id=RPPYCLGHNYZG5P3ODMENLIY3O76NX7LY5LF3OVPC73WUTXKPGUNA, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.698036912 (569h0m41.569119957s) [noise=true , address=145.79.227.29:4943   , piece_id=WJ53ZHRY7AVAPLEOC6YIGX2AABSGXWBZFNIFK4GYOAPUR5IS3O7Q, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.698200413 (569h0m41.569283448s) [noise=true , address=145.79.227.17:4971   , piece_id=UTD2HONOAADZBH2SIRTPRAKHOC4DXN2VYL4FESZODSZJITGTZLPQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.698631725 (569h0m41.56971477s) [noise=true , address=145.79.227.18:4007   , piece_id=EBNOQQ7GF5JLFNLKWUVA5GQLHIKSU3OF6WRPBOMUQM7QTU25MOLA, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.699071301 (569h0m41.570154346s) [noise=true , address=77.90.31.16:4735     , piece_id=JNMFBEE4JP2XLMAIHX244L32F6F7NFECXAPJ2I6ZAFOG45JKLLJQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.699159289 (569h0m41.570242334s) [noise=true , address=77.90.31.19:4841     , piece_id=Z4PBKBE63IPGIF676664EURPS6HJ4QN4VXITS6DCNZYWVNMKDTLQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.699553700 (569h0m41.570636745s) [noise=true , address=45.91.116.13:4155    , piece_id=VE7PFAF7WX2IMDH2E3ZPEYXEXKS74N55BDSCGHIPYLF2UOJ7RQIQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.699942251 (569h0m41.571025306s) [noise=true , address=45.91.116.8:4046     , piece_id=3ZBYO3G6B2GK5TBUZQSHSIZ2PL2ILCGAYBBZSHPP47QAZWAGWNVA, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.699950977 (569h0m41.571034022s) [noise=true , address=145.79.227.42:4123   , piece_id=EHMAUQCSAZAKAJ54YD4WSOI6EF2IL77XJVJXJQFT4BFL4C7ZHY5A, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.700566118 (569h0m41.571649163s) [noise=true , address=45.91.116.10:4084    , piece_id=QG5CKWLMPJJZYGDWWY2Y32LGQI7HSZ3RJ4EXOSTNYQI4LGDISH2Q, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.701213090 (569h0m41.572296125s) [noise=true , address=145.79.227.46:4603   , piece_id=NYGAPWWDTXJZH47O36TT3Q5QVAM5HVSJBME33JQW6GTMOZNT5NRQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.702075362 (569h0m41.573158407s) [noise=true , address=145.79.227.2:4115    , piece_id=5WHP2HRUNOGUXPAY5UCIAUP2OGXDHYWIUHXC3LC7XGK67EC7BD5A, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.703211916 (569h0m41.574294961s) [noise=true , address=145.79.227.47:4266   , piece_id=U65JZ24SHR3J74CUGS6YMPXDPTEOORVBLMLY5YDFMUWLL5RZT3QQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.703850202 (569h0m41.574933237s) [noise=true , address=145.79.227.16:4912   , piece_id=5CKINPZ6MS33TFVJWQDO4L6TOGET3HPICO3WCV75HVOKAUJBGSKQ, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.704499818 (569h0m41.575582863s) [noise=true , address=145.79.227.5:4221    , piece_id=4YQLQLZDFBKHWPK5KFYZ6RSWVR4W4HNN5IIBO4WPJL4HDGMST5DA, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:41.704728274 (569h0m41.575811309s) [noise=true , address=145.79.227.45:4231   , piece_id=R2MUJ66SFLIBKWTTDALXCDMQUCOTZRY2AUG5NFO4XSET6JHR7REA, seg_pos={0 0}, upload=498874]: Uploading piece...
Dec 29 08:29:43.404789886 (569h0m43.275872971s) [upload=498874]: Next returned nil segment
Dec 29 08:29:43.404849430 (569h0m43.275932515s) [upload=498874]: Waiting for error group managing segments...
Dec 29 08:29:43.701464660 (569h0m43.572547695s) [noise=true , address=145.79.227.46:4603   , piece_id=NYGAPWWDTXJZH47O36TT3Q5QVAM5HVSJBME33JQW6GTMOZNT5NRQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.705316515 (569h0m43.57639955s) [noise=true , address=145.79.227.47:4266   , piece_id=U65JZ24SHR3J74CUGS6YMPXDPTEOORVBLMLY5YDFMUWLL5RZT3QQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.705528408 (569h0m43.576611443s) [noise=true , address=145.79.227.5:4221    , piece_id=4YQLQLZDFBKHWPK5KFYZ6RSWVR4W4HNN5IIBO4WPJL4HDGMST5DA, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.706179637 (569h0m43.577262683s) [noise=true , address=145.79.227.16:4912   , piece_id=5CKINPZ6MS33TFVJWQDO4L6TOGET3HPICO3WCV75HVOKAUJBGSKQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.707041469 (569h0m43.578124544s) [noise=true , address=145.79.227.42:4123   , piece_id=EHMAUQCSAZAKAJ54YD4WSOI6EF2IL77XJVJXJQFT4BFL4C7ZHY5A, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.707075604 (569h0m43.578158639s) [noise=true , address=145.79.227.2:4115    , piece_id=5WHP2HRUNOGUXPAY5UCIAUP2OGXDHYWIUHXC3LC7XGK67EC7BD5A, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.707835812 (569h0m43.578918857s) [noise=true , address=145.79.227.17:4971   , piece_id=UTD2HONOAADZBH2SIRTPRAKHOC4DXN2VYL4FESZODSZJITGTZLPQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.714815982 (569h0m43.585899018s) [noise=true , address=145.79.227.45:4231   , piece_id=R2MUJ66SFLIBKWTTDALXCDMQUCOTZRY2AUG5NFO4XSET6JHR7REA, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.720314713 (569h0m43.591397748s) [noise=true , address=145.79.227.18:4007   , piece_id=EBNOQQ7GF5JLFNLKWUVA5GQLHIKSU3OF6WRPBOMUQM7QTU25MOLA, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.882343410 (569h0m43.753426445s) [noise=true , address=77.90.31.17:4773     , piece_id=RPPYCLGHNYZG5P3ODMENLIY3O76NX7LY5LF3OVPC73WUTXKPGUNA, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.923752269 (569h0m43.794835314s) [noise=true , address=45.91.116.10:4084    , piece_id=QG5CKWLMPJJZYGDWWY2Y32LGQI7HSZ3RJ4EXOSTNYQI4LGDISH2Q, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:43.979013204 (569h0m43.850096239s) [noise=true , address=45.91.116.8:4046     , piece_id=3ZBYO3G6B2GK5TBUZQSHSIZ2PL2ILCGAYBBZSHPP47QAZWAGWNVA, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:44.027001361 (569h0m43.898084396s) [noise=true , address=45.91.116.13:4155    , piece_id=VE7PFAF7WX2IMDH2E3ZPEYXEXKS74N55BDSCGHIPYLF2UOJ7RQIQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: <nil>
Dec 29 08:29:44.027028953 (569h0m43.898111998s) [seg_pos={0 0}, upload=498874]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:29:44.027653773 (569h0m43.898736808s) [noise=true , address=77.90.31.16:4735     , piece_id=JNMFBEE4JP2XLMAIHX244L32F6F7NFECXAPJ2I6ZAFOG45JKLLJQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF
Dec 29 08:29:44.027672289 (569h0m43.898755324s) [noise=true , address=77.90.31.16:4735     , piece_id=JNMFBEE4JP2XLMAIHX244L32F6F7NFECXAPJ2I6ZAFOG45JKLLJQ, seg_pos={0 0}, upload=498874]: Long tail cancellation due to optimal threshold
Dec 29 08:29:44.027691715 (569h0m43.89877475s) [noise=true , address=145.79.227.29:4943   , piece_id=WJ53ZHRY7AVAPLEOC6YIGX2AABSGXWBZFNIFK4GYOAPUR5IS3O7Q, seg_pos={0 0}, upload=498874]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fc5Trj8XxacZhzxdksUVT5CbG6uTmiwdUPjreYi4UMPY1TXWz): protocol: expected piece hash; context canceled; EOF
Dec 29 08:29:44.027701955 (569h0m43.898785s) [noise=true , address=145.79.227.29:4943   , piece_id=WJ53ZHRY7AVAPLEOC6YIGX2AABSGXWBZFNIFK4GYOAPUR5IS3O7Q, seg_pos={0 0}, upload=498874]: Long tail cancellation due to optimal threshold
Dec 29 08:29:44.027740088 (569h0m43.898823133s) [noise=true , address=77.90.31.19:4841     , piece_id=Z4PBKBE63IPGIF676664EURPS6HJ4QN4VXITS6DCNZYWVNMKDTLQ, seg_pos={0 0}, upload=498874]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:29:44.027750858 (569h0m43.898833893s) [noise=true , address=77.90.31.19:4841     , piece_id=Z4PBKBE63IPGIF676664EURPS6HJ4QN4VXITS6DCNZYWVNMKDTLQ, seg_pos={0 0}, upload=498874]: Long tail cancellation due to optimal threshold
Dec 29 08:29:44.027809029 (569h0m43.898892064s) [seg_pos={0 0}, upload=498874]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:29:44.027942213 (569h0m43.899025258s) [upload=498874]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:29:44.027982159 (569h0m43.899065194s) [upload=498874]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:29:44.314172367 (569h0m44.185255412s) [upload=498874]: Done uploading segments...
Dec 29 08:29:44.314196393 (569h0m44.185279438s) [upload=498874]: Upload finished. err: <nil>
Duration: 30.004525198s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [......................X..X......X..........X.X..X..X.......X]
duration: 
5.001428233s
trace: 
https://jaeger.dev.storj.io/trace/4ea4b41656f8a3e1
FAIL (2025-12-29 08:29:47 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001449093s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [.......................X..X......X..........X.X..X..X.......]
duration: 
3.556191673s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:29:13 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.556218886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [.......X...X............XX....X.X..X...X..X...X...X...X.....]
duration: 
5.72691001s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:29:09 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 5.726942843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..........X..........................X......................]
duration: 
31.566187559s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:29:05 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 31.566206135s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [........X...X............XX....X.X..X...X..X...X...X...X....]
duration: 
5.664819159s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:28:37 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 5.664845558s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...........X..........................X.....................]
duration: 
37.836612076s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:28:33 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 37.836641413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [........................X..X......X..........X.X..X..X......]
duration: 
4.818604167s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:28:05 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.818631127s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [X........X...X............XX....X.X..X...X..X...X...X...X...]
duration: 
5.93780309s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:27:44 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 5.937828078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [............X..........................X....................]
duration: 
19.911635365s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:27:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 19.911662847s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.............X..........................X...................]
duration: 
15.475130626s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:26:58 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 15.475144753s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [X........................X..X......X..........X.X..X..X.....]
duration: 
3.534839456s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:26:52 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.534890414s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [X.............X..........................X..................]
duration: 
39.661971249s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:26:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 39.662001537s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.X........X...X............XX....X.X..X...X..X...X...X...X..]
duration: 
6.012419031s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:26:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 6.012445902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [X.X........X...X............XX....X.X..X...X..X...X...X...X.]
duration: 
5.905917534s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:26:14 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 5.905942213s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
15 out of 60 is failed (threshold=3) [XX.X........X...X............XX....X.X..X...X..X...X...X...X]
duration: 
30.00321778s
trace: 
https://jaeger.dev.storj.io/trace/7cb98379a01fd61d
FAIL (2025-12-29 08:25:40 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, Piece ID: VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A) context deadline exceeded; piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A) context deadline exceeded
======LOGS======
9 08:25:11.547304946 (568h56m11.418387991s) [upload=498808]: Starting upload
Dec 29 08:25:11.547393905 (568h56m11.41847698s) [upload=498808]: Done starting upload
Dec 29 08:25:11.547722441 (568h56m11.418805486s) [upload=498808]: Uploading segments...
Dec 29 08:25:11.547766906 (568h56m11.418849961s) [upload=498808]: Got next segment. Inline: false
Dec 29 08:25:11.547812121 (568h56m11.418895156s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:25:11.547816761 (568h56m11.418899796s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:25:11.791971298 (568h56m11.663054343s) [seg_pos={0 0}, upload=498808]: Begin upload segment...
Dec 29 08:25:11.792018818 (568h56m11.663101853s) [seg_pos={0 0}, upload=498808]: Done begin upload segment.
Dec 29 08:25:11.792056119 (568h56m11.663139164s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792158043 (568h56m11.663241078s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.792994015 (568h56m11.66407711s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793670843 (568h56m11.664753878s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.793908917 (568h56m11.664991952s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794274784 (568h56m11.665357829s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794404630 (568h56m11.665487675s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794911726 (568h56m11.665994761s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.794969415 (568h56m11.666052451s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795364008 (568h56m11.666447053s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795648138 (568h56m11.666731183s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.795882956 (568h56m11.666966001s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796345376 (568h56m11.667428411s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796358340 (568h56m11.667441385s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796783110 (568h56m11.667866155s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:11.796901024 (568h56m11.667984069s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Uploading piece...
Dec 29 08:25:13.484835286 (568h56m13.355918331s) [upload=498808]: Next returned nil segment
Dec 29 08:25:13.484849943 (568h56m13.355932998s) [upload=498808]: Waiting for error group managing segments...
Dec 29 08:25:13.777303920 (568h56m13.648386965s) [noise=true , address=145.79.227.12:4767   , piece_id=ZZH4ZSHZRBD3X4CQXUQM6L2XFOZYBVHA2OPAPG5BJBAMWRSELGFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.777953567 (568h56m13.649036602s) [noise=true , address=145.79.227.7:4295    , piece_id=7BDQNOCWXBUPTSIHNAAUUAQTV47UI4PEQIXKJDOQHBTB37HODIBA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.783163377 (568h56m13.654246412s) [noise=true , address=145.79.227.48:4302   , piece_id=ZXNFC67CNPOHMCJQGASA7JZDBSTANAPMPBOIAVNWGYBKFMNTDZLA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.788548401 (568h56m13.659631446s) [noise=true , address=145.79.227.18:4006   , piece_id=DVU2VCTQKMV2H67S57XNJ2P3WKZURX2ILZXPAB6667EVSNNSL2XA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.793899880 (568h56m13.664982915s) [noise=true , address=145.79.227.17:4972   , piece_id=44QXZBPS2OLTBHBYGARU4TNRUWDFAROYU27OZLYEYDACAPHSTHCQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.799863726 (568h56m13.670946761s) [noise=true , address=145.79.227.15:4873   , piece_id=2DPGA6UL5R6NLNWNM3Z6VZOCIILVBNHZYX5SESJLE7EB7TCFJBOA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.804091948 (568h56m13.675174993s) [noise=true , address=145.79.227.5:4221    , piece_id=FTN7JRBJNXGJJIDQR33W7VXRO56ISGT5G2QZ2EGBPXRLSALUGUDA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:13.805279079 (568h56m13.676362114s) [noise=true , address=145.79.227.3:4150    , piece_id=BKQMAAL4Z2EHYQ25YQ6MDQCFOXISGO4GB662YPRP7ZG4LTRDIRMQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.055676844 (568h56m13.926759879s) [noise=true , address=45.91.116.10:4081    , piece_id=VBCVPHVGE5TYJGX2FVEOSI5UXY3VMVC6YGIEH774IUUQIXZKO73A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.071179043 (568h56m13.942262078s) [noise=true , address=45.91.116.7:4012     , piece_id=7KEEU46ROHZ4NNITN2FSQDY5ZN7YNUG6CRVLWKACD7W6OTGPVM7Q, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.074583194 (568h56m13.945666229s) [noise=true , address=45.91.116.6:4975     , piece_id=DZYS4D5O526YUYAGKP5NPQALZ5KIE2K4SS26VT645WP7MEPZ4K2A, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.102985376 (568h56m13.974068421s) [noise=true , address=77.90.31.15:4700     , piece_id=KOKGBTC37WCJSLY4UFIGWFAQQIJYFENITUNMJF6FW5ZP5UJCLOSA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722027585 (568h56m14.59311063s) [noise=true , address=45.91.116.4:4939     , piece_id=PYFIT3NZCESUQIN4NFGARLE6X6BKISTINQW6Z3MP2GMT6DJMZEDQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: <nil>
Dec 29 08:25:14.722051521 (568h56m14.593134556s) [seg_pos={0 0}, upload=498808]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:25:14.722361321 (568h56m14.593444356s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54402->45.91.116.8:4046: use of closed network connection
Dec 29 08:25:14.722371590 (568h56m14.593454655s) [noise=true , address=45.91.116.8:4046     , piece_id=PQXEBLR5BYLW2E4VCZGLA5V6Z6NHS5KWDCIYA7DYWFJ4MHQRTADA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722572222 (568h56m14.593655267s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722603382 (568h56m14.593686427s) [noise=true , address=45.91.116.12:4228    , piece_id=R3UCZ64OUWPDC2XMIFQPTPJ55TK64P7X7R55FYXWCSTDHYMOQJFA, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722784947 (568h56m14.593867992s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Dec 29 08:25:14.722792942 (568h56m14.593875977s) [noise=true , address=145.79.227.34:4905   , piece_id=TG5BCVXUH7R5DQSBIZX2CI5LDTGWNWPHZJN5PQFNNLKCEX733EQQ, seg_pos={0 0}, upload=498808]: Long tail cancellation due to optimal threshold
Dec 29 08:25:14.722811849 (568h56m14.593894884s) [seg_pos={0 0}, upload=498808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:25:14.722894365 (568h56m14.59397741s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:25:14.722899886 (568h56m14.593982921s) [upload=498808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:25:15.029601692 (568h56m14.900684737s) [upload=498808]: Done uploading segments...
Dec 29 08:25:15.029627832 (568h56m14.900710867s) [upload=498808]: Upload finished. err: <nil>
Duration: 30.003247878s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [.X........................X..X......X..........X.X..X..X....]
duration: 
3.472270054s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:25:39 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.472299339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [.X.............X..........................X.................]
duration: 
15.198543697s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:25:29 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 15.198573072s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [..X........................X..X......X..........X.X..X..X...]
duration: 
3.399779993s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:25:18 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.399813055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [X.X.............X..........................X................]
duration: 
15.401044057s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:25:12 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 15.401068954s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
4 out of 60 is failed (threshold=3) [.X.X.............X..........................X...............]
duration: 
36.643489044s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:24:56 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 36.643505705s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
15 out of 60 is failed (threshold=3) [XXX.X........X...X............XX....X.X..X...X..X...X...X...]
duration: 
5.731561475s
trace: 
https://jaeger.dev.storj.io/trace/240cb6a4ff643c55
FAIL (2025-12-29 08:24:48 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ec 29 08:22:38.899704177 (568h53m38.770787223s) [upload=498766]: Starting upload
Dec 29 08:22:38.899778168 (568h53m38.770861203s) [upload=498766]: Done starting upload
Dec 29 08:22:38.900085474 (568h53m38.771168529s) [upload=498766]: Uploading segments...
Dec 29 08:22:38.900211574 (568h53m38.771294639s) [upload=498766]: Got next segment. Inline: false
Dec 29 08:22:38.900277089 (568h53m38.771360124s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:22:38.900285815 (568h53m38.77136885s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:22:39.155715779 (568h53m39.026798824s) [seg_pos={0 0}, upload=498766]: Begin upload segment...
Dec 29 08:22:39.155757589 (568h53m39.026840624s) [seg_pos={0 0}, upload=498766]: Done begin upload segment.
Dec 29 08:22:39.155782887 (568h53m39.026865922s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.157730997 (568h53m39.028814042s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.158309538 (568h53m39.029392583s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159507089 (568h53m39.030590134s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159792572 (568h53m39.030875617s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.160565053 (568h53m39.031648088s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161336331 (568h53m39.032419366s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161589795 (568h53m39.03267284s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.162993738 (568h53m39.034076783s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.164365449 (568h53m39.035448534s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.165720981 (568h53m39.036804026s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.166392609 (568h53m39.037475644s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.167487755 (568h53m39.03857082s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.168324889 (568h53m39.039407924s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.169249419 (568h53m39.040332454s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.170568551 (568h53m39.041651586s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:40.858833741 (568h53m40.729916776s) [upload=498766]: Next returned nil segment
Dec 29 08:22:40.858847026 (568h53m40.729930061s) [upload=498766]: Waiting for error group managing segments...
Dec 29 08:22:41.144242128 (568h53m41.015325163s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.145991288 (568h53m41.017074334s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.149224355 (568h53m41.0203074s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.164606073 (568h53m41.035689109s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.167081888 (568h53m41.038164933s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170495087 (568h53m41.041578122s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170887344 (568h53m41.041970409s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.178695803 (568h53m41.049778838s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.381461526 (568h53m41.252544561s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.431006728 (568h53m41.302089763s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.458925168 (568h53m41.330008213s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.529137431 (568h53m41.400220476s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650045037 (568h53m41.521128072s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650065976 (568h53m41.521149011s) [seg_pos={0 0}, upload=498766]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:22:41.650243264 (568h53m41.521326309s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection
Dec 29 08:22:41.650249026 (568h53m41.521332061s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650320361 (568h53m41.521403396s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650330460 (568h53m41.521413495s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650601827 (568h53m41.521684882s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650613689 (568h53m41.521696725s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650638267 (568h53m41.521721302s) [seg_pos={0 0}, upload=498766]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:22:41.650710073 (568h53m41.521793118s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:22:41.650714682 (568h53m41.521797717s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:22:41.937840108 (568h53m41.808923153s) [upload=498766]: Done uploading segments...
Dec 29 08:22:41.937867781 (568h53m41.808950826s) [upload=498766]: Upload finished. err: <nil>
Duration: 5.731661745s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [...X........................X..X......X..........X.X..X..X..]
duration: 
3.496394652s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:24:42 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.496444036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [XXXX.X........X...X............XX....X.X..X...X..X...X...X..]
duration: 
6.05242644s
trace: 
https://jaeger.dev.storj.io/trace/240cb6a4ff643c55
FAIL (2025-12-29 08:24:24 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ec 29 08:22:38.899704177 (568h53m38.770787223s) [upload=498766]: Starting upload
Dec 29 08:22:38.899778168 (568h53m38.770861203s) [upload=498766]: Done starting upload
Dec 29 08:22:38.900085474 (568h53m38.771168529s) [upload=498766]: Uploading segments...
Dec 29 08:22:38.900211574 (568h53m38.771294639s) [upload=498766]: Got next segment. Inline: false
Dec 29 08:22:38.900277089 (568h53m38.771360124s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:22:38.900285815 (568h53m38.77136885s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:22:39.155715779 (568h53m39.026798824s) [seg_pos={0 0}, upload=498766]: Begin upload segment...
Dec 29 08:22:39.155757589 (568h53m39.026840624s) [seg_pos={0 0}, upload=498766]: Done begin upload segment.
Dec 29 08:22:39.155782887 (568h53m39.026865922s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.157730997 (568h53m39.028814042s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.158309538 (568h53m39.029392583s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159507089 (568h53m39.030590134s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159792572 (568h53m39.030875617s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.160565053 (568h53m39.031648088s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161336331 (568h53m39.032419366s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161589795 (568h53m39.03267284s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.162993738 (568h53m39.034076783s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.164365449 (568h53m39.035448534s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.165720981 (568h53m39.036804026s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.166392609 (568h53m39.037475644s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.167487755 (568h53m39.03857082s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.168324889 (568h53m39.039407924s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.169249419 (568h53m39.040332454s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.170568551 (568h53m39.041651586s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:40.858833741 (568h53m40.729916776s) [upload=498766]: Next returned nil segment
Dec 29 08:22:40.858847026 (568h53m40.729930061s) [upload=498766]: Waiting for error group managing segments...
Dec 29 08:22:41.144242128 (568h53m41.015325163s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.145991288 (568h53m41.017074334s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.149224355 (568h53m41.0203074s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.164606073 (568h53m41.035689109s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.167081888 (568h53m41.038164933s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170495087 (568h53m41.041578122s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170887344 (568h53m41.041970409s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.178695803 (568h53m41.049778838s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.381461526 (568h53m41.252544561s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.431006728 (568h53m41.302089763s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.458925168 (568h53m41.330008213s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.529137431 (568h53m41.400220476s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650045037 (568h53m41.521128072s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650065976 (568h53m41.521149011s) [seg_pos={0 0}, upload=498766]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:22:41.650243264 (568h53m41.521326309s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection
Dec 29 08:22:41.650249026 (568h53m41.521332061s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650320361 (568h53m41.521403396s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650330460 (568h53m41.521413495s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650601827 (568h53m41.521684882s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650613689 (568h53m41.521696725s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650638267 (568h53m41.521721302s) [seg_pos={0 0}, upload=498766]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:22:41.650710073 (568h53m41.521793118s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:22:41.650714682 (568h53m41.521797717s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:22:41.937840108 (568h53m41.808923153s) [upload=498766]: Done uploading segments...
Dec 29 08:22:41.937867781 (568h53m41.808950826s) [upload=498766]: Upload finished. err: <nil>
Duration: 6.052450404s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [X.X.X.............X..........................X..............]
duration: 
42.416354384s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:24:19 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 42.416379391s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [....X........................X..X......X..........X.X..X..X.]
duration: 
3.563628922s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:23:54 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.563652226s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.X.X.X.............X..........................X.............]
duration: 
16.582952927s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:23:33 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 16.58299064s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [.XXXX.X........X...X............XX....X.X..X...X..X...X...X.]
duration: 
5.902679645s
trace: 
https://jaeger.dev.storj.io/trace/240cb6a4ff643c55
FAIL (2025-12-29 08:23:17 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ec 29 08:22:38.899704177 (568h53m38.770787223s) [upload=498766]: Starting upload
Dec 29 08:22:38.899778168 (568h53m38.770861203s) [upload=498766]: Done starting upload
Dec 29 08:22:38.900085474 (568h53m38.771168529s) [upload=498766]: Uploading segments...
Dec 29 08:22:38.900211574 (568h53m38.771294639s) [upload=498766]: Got next segment. Inline: false
Dec 29 08:22:38.900277089 (568h53m38.771360124s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:22:38.900285815 (568h53m38.77136885s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:22:39.155715779 (568h53m39.026798824s) [seg_pos={0 0}, upload=498766]: Begin upload segment...
Dec 29 08:22:39.155757589 (568h53m39.026840624s) [seg_pos={0 0}, upload=498766]: Done begin upload segment.
Dec 29 08:22:39.155782887 (568h53m39.026865922s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.157730997 (568h53m39.028814042s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.158309538 (568h53m39.029392583s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159507089 (568h53m39.030590134s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159792572 (568h53m39.030875617s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.160565053 (568h53m39.031648088s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161336331 (568h53m39.032419366s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161589795 (568h53m39.03267284s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.162993738 (568h53m39.034076783s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.164365449 (568h53m39.035448534s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.165720981 (568h53m39.036804026s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.166392609 (568h53m39.037475644s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.167487755 (568h53m39.03857082s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.168324889 (568h53m39.039407924s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.169249419 (568h53m39.040332454s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.170568551 (568h53m39.041651586s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:40.858833741 (568h53m40.729916776s) [upload=498766]: Next returned nil segment
Dec 29 08:22:40.858847026 (568h53m40.729930061s) [upload=498766]: Waiting for error group managing segments...
Dec 29 08:22:41.144242128 (568h53m41.015325163s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.145991288 (568h53m41.017074334s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.149224355 (568h53m41.0203074s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.164606073 (568h53m41.035689109s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.167081888 (568h53m41.038164933s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170495087 (568h53m41.041578122s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170887344 (568h53m41.041970409s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.178695803 (568h53m41.049778838s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.381461526 (568h53m41.252544561s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.431006728 (568h53m41.302089763s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.458925168 (568h53m41.330008213s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.529137431 (568h53m41.400220476s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650045037 (568h53m41.521128072s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650065976 (568h53m41.521149011s) [seg_pos={0 0}, upload=498766]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:22:41.650243264 (568h53m41.521326309s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection
Dec 29 08:22:41.650249026 (568h53m41.521332061s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650320361 (568h53m41.521403396s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650330460 (568h53m41.521413495s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650601827 (568h53m41.521684882s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650613689 (568h53m41.521696725s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650638267 (568h53m41.521721302s) [seg_pos={0 0}, upload=498766]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:22:41.650710073 (568h53m41.521793118s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:22:41.650714682 (568h53m41.521797717s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:22:41.937840108 (568h53m41.808923153s) [upload=498766]: Done uploading segments...
Dec 29 08:22:41.937867781 (568h53m41.808950826s) [upload=498766]: Upload finished. err: <nil>
Duration: 5.90270778s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
17 out of 60 is failed (threshold=3) [X.XXXX.X........X...X............XX....X.X..X...X..X...X...X]
duration: 
30.005316393s
trace: 
https://jaeger.dev.storj.io/trace/240cb6a4ff643c55
FAIL (2025-12-29 08:23:08 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ec 29 08:22:38.899704177 (568h53m38.770787223s) [upload=498766]: Starting upload
Dec 29 08:22:38.899778168 (568h53m38.770861203s) [upload=498766]: Done starting upload
Dec 29 08:22:38.900085474 (568h53m38.771168529s) [upload=498766]: Uploading segments...
Dec 29 08:22:38.900211574 (568h53m38.771294639s) [upload=498766]: Got next segment. Inline: false
Dec 29 08:22:38.900277089 (568h53m38.771360124s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:22:38.900285815 (568h53m38.77136885s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:22:39.155715779 (568h53m39.026798824s) [seg_pos={0 0}, upload=498766]: Begin upload segment...
Dec 29 08:22:39.155757589 (568h53m39.026840624s) [seg_pos={0 0}, upload=498766]: Done begin upload segment.
Dec 29 08:22:39.155782887 (568h53m39.026865922s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.157730997 (568h53m39.028814042s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.158309538 (568h53m39.029392583s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159507089 (568h53m39.030590134s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.159792572 (568h53m39.030875617s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.160565053 (568h53m39.031648088s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161336331 (568h53m39.032419366s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.161589795 (568h53m39.03267284s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.162993738 (568h53m39.034076783s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.164365449 (568h53m39.035448534s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.165720981 (568h53m39.036804026s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.166392609 (568h53m39.037475644s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.167487755 (568h53m39.03857082s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.168324889 (568h53m39.039407924s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.169249419 (568h53m39.040332454s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:39.170568551 (568h53m39.041651586s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Uploading piece...
Dec 29 08:22:40.858833741 (568h53m40.729916776s) [upload=498766]: Next returned nil segment
Dec 29 08:22:40.858847026 (568h53m40.729930061s) [upload=498766]: Waiting for error group managing segments...
Dec 29 08:22:41.144242128 (568h53m41.015325163s) [noise=true , address=145.79.227.6:4258    , piece_id=P3QOHZBZOPABLWJP6BORQEPECS5SBEUA2K4XUUGNYHGIOWORTZJA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.145991288 (568h53m41.017074334s) [noise=true , address=145.79.227.5:4221    , piece_id=BJOMSUB2ZEC7TUX2EQCGYP52SWNG6JFFN4BPQVEP2ZRZF2H75HRQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.149224355 (568h53m41.0203074s) [noise=true , address=145.79.227.46:4601   , piece_id=IHX4MSWIPYJH65G4UO3GR4TN2FQH32FOKGMZXN47M2M7SWKO744Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.164606073 (568h53m41.035689109s) [noise=true , address=145.79.227.15:4873   , piece_id=GLDMTMMQOLSA27URH5R4WZQCVKKIEXPMJ53BPATTGL7ZYCNOXYCQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.167081888 (568h53m41.038164933s) [noise=true , address=145.79.227.7:4294    , piece_id=NBKDXV3Z3RYAXYGDVNETZ4QN4GCCBEGGFKAGDFU2YXTN23KXAGOA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170495087 (568h53m41.041578122s) [noise=true , address=145.79.227.52:4445   , piece_id=ATWA266GREK7KX3S7CL7GDSSGWYC47Z2Y6REEUWH47JKNBDYNAEA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.170887344 (568h53m41.041970409s) [noise=true , address=145.79.227.48:4302   , piece_id=SE6TSP5MJ73FSF3OW62DWBLYFRCRIXTZDKHCSFLQRSGPC4OKFNNA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.178695803 (568h53m41.049778838s) [noise=true , address=145.79.227.25:4619   , piece_id=U7RXU3YL7MPNGBSLQ6JI5BBKY3ZFFPKP5BTNT5YJMY35QPMPHOKA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.381461526 (568h53m41.252544561s) [noise=true , address=77.90.31.17:4773     , piece_id=5QJPNSF2RBD2IQEBRTK3KXL4SKWMU7AABR2QPQ6VRM65OS4U2QWQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.431006728 (568h53m41.302089763s) [noise=true , address=45.91.116.14:4191    , piece_id=MOAEDGYPZ3ORDAMC5FQH35EGDD3TAUWTGEA4D73GDOZUPGMUTW7A, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.458925168 (568h53m41.330008213s) [noise=true , address=77.90.31.21:4880     , piece_id=B3LBACU55JPA4OWFXAE6CBPQY7K4NJVBJARH6F2P2OMIRT2XVPEQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.529137431 (568h53m41.400220476s) [noise=true , address=45.91.116.5:4297     , piece_id=3T5I4D34FAMMJMLYZ4QBHTJNMEBD4F6BU3OFOWNM6RXTHWEHLJ2Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650045037 (568h53m41.521128072s) [noise=true , address=45.91.116.6:4976     , piece_id=7MCPCCGAHXVTCI4FK3DMBXX6GLOYXO7LWEC5JMR372KIX7QYQ3YQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: <nil>
Dec 29 08:22:41.650065976 (568h53m41.521149011s) [seg_pos={0 0}, upload=498766]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:22:41.650243264 (568h53m41.521326309s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54104->77.90.31.12:4589: use of closed network connection
Dec 29 08:22:41.650249026 (568h53m41.521332061s) [noise=true , address=77.90.31.12:4589     , piece_id=P77IJBID544XT4QXSCQEOK35TMKFXBIPOPU5YFI4EKBGC7BIFA5Q, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650320361 (568h53m41.521403396s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650330460 (568h53m41.521413495s) [noise=true , address=45.91.116.4:4937     , piece_id=OJMHRUML2XE3J5FOOGHI3UQOFXCNBA27BZKOKEOE5AP23M37W2UA, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650601827 (568h53m41.521684882s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF
Dec 29 08:22:41.650613689 (568h53m41.521696725s) [noise=true , address=45.91.116.10:4084    , piece_id=BZN4VTSCG7KMAFX7765GYF3U37PESZMNNDJIXSRMGVZHRWNG2ZPQ, seg_pos={0 0}, upload=498766]: Long tail cancellation due to optimal threshold
Dec 29 08:22:41.650638267 (568h53m41.521721302s) [seg_pos={0 0}, upload=498766]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:22:41.650710073 (568h53m41.521793118s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:22:41.650714682 (568h53m41.521797717s) [upload=498766]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:22:41.937840108 (568h53m41.808923153s) [upload=498766]: Done uploading segments...
Dec 29 08:22:41.937867781 (568h53m41.808950826s) [upload=498766]: Upload finished. err: <nil>
Duration: 30.005337574s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [..X.X.X.............X..........................X............]
duration: 
29.257709659s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:23:04 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 29.257727102s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [.....X........................X..X......X..........X.X..X..X]
duration: 
5.001942059s
trace: 
https://jaeger.dev.storj.io/trace/6a5d4873a094e79b
FAIL (2025-12-29 08:22:53 UTC) 
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001964592s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [...X.X.X.............X..........................X...........]
duration: 
29.394564191s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:22:35 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 29.394637683s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [....X.X.X.............X..........................X..........]
duration: 
39.843854688s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:22:03 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 39.843877862s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [.X.XXXX.X........X...X............XX....X.X..X...X..X...X...]
duration: 
9.080099325s
trace: 
https://jaeger.dev.storj.io/trace/65ef3e3f946e21d4
FAIL (2025-12-29 08:22:00 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA) context deadline exceeded; piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ) context deadline exceeded
======LOGS======
Dec 29 08:20:22.760401777 (568h51m22.631484823s) [upload=498728]: Starting upload
Dec 29 08:20:22.760484224 (568h51m22.631567269s) [upload=498728]: Done starting upload
Dec 29 08:20:22.760570930 (568h51m22.631653975s) [upload=498728]: Uploading segments...
Dec 29 08:20:22.760731294 (568h51m22.63181434s) [upload=498728]: Got next segment. Inline: false
Dec 29 08:20:22.760775670 (568h51m22.631858715s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:20:22.760782372 (568h51m22.631865417s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:20:23.004266840 (568h51m22.875349895s) [seg_pos={0 0}, upload=498728]: Begin upload segment...
Dec 29 08:20:23.004343656 (568h51m22.875426701s) [seg_pos={0 0}, upload=498728]: Done begin upload segment.
Dec 29 08:20:23.004402989 (568h51m22.875486044s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.004471198 (568h51m22.875554244s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005433692 (568h51m22.876516737s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005984681 (568h51m22.877067726s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007095807 (568h51m22.878178852s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007253186 (568h51m22.878336221s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.008748694 (568h51m22.879831739s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009486750 (568h51m22.880569795s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009772374 (568h51m22.880855419s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.010493155 (568h51m22.8815762s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011140649 (568h51m22.882223694s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011589233 (568h51m22.882672268s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.012028108 (568h51m22.883111163s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014337595 (568h51m22.88542063s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014850152 (568h51m22.885933197s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.015257748 (568h51m22.886340793s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:24.618931135 (568h51m24.49001417s) [upload=498728]: Next returned nil segment
Dec 29 08:20:24.618949691 (568h51m24.490032736s) [upload=498728]: Waiting for error group managing segments...
Dec 29 08:20:25.000682572 (568h51m24.871765607s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001062866 (568h51m24.872145921s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001363920 (568h51m24.872446965s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.002695034 (568h51m24.873778079s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004076094 (568h51m24.875159129s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004874123 (568h51m24.875957158s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.019135710 (568h51m24.890218745s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.021700393 (568h51m24.892783428s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.024934521 (568h51m24.896017566s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.284220667 (568h51m25.155303712s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.310715378 (568h51m25.181798413s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.312813422 (568h51m25.183896468s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314297409 (568h51m25.185380444s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314307217 (568h51m25.185390252s) [seg_pos={0 0}, upload=498728]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:20:25.314502258 (568h51m25.185585293s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314510083 (568h51m25.185593119s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314738728 (568h51m25.185821763s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314745050 (568h51m25.185828095s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314912980 (568h51m25.185996015s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: read tcp 188.166.225.170:40388->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 29 08:20:25.314918260 (568h51m25.186001295s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314931205 (568h51m25.18601424s) [seg_pos={0 0}, upload=498728]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:20:25.314994957 (568h51m25.186077992s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:20:25.315000467 (568h51m25.186083512s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:20:25.600705118 (568h51m25.471788153s) [upload=498728]: Done uploading segments...
Dec 29 08:20:25.600725036 (568h51m25.471808071s) [upload=498728]: Upload finished. err: <nil>
Duration: 9.080131184s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [......X........................X..X......X..........X.X..X..]
duration: 
3.563330945s
trace: 
https://jaeger.dev.storj.io/trace/7072b84d0cb8ae76
FAIL (2025-12-29 08:21:57 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.563357444s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.....X.X.X.............X..........................X.........]
duration: 
35.391487182s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:21:22 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 35.391516629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [..X.XXXX.X........X...X............XX....X.X..X...X..X...X..]
duration: 
6.035265469s
trace: 
https://jaeger.dev.storj.io/trace/65ef3e3f946e21d4
FAIL (2025-12-29 08:21:19 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA) context deadline exceeded; piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ) context deadline exceeded
======LOGS======
Dec 29 08:20:22.760401777 (568h51m22.631484823s) [upload=498728]: Starting upload
Dec 29 08:20:22.760484224 (568h51m22.631567269s) [upload=498728]: Done starting upload
Dec 29 08:20:22.760570930 (568h51m22.631653975s) [upload=498728]: Uploading segments...
Dec 29 08:20:22.760731294 (568h51m22.63181434s) [upload=498728]: Got next segment. Inline: false
Dec 29 08:20:22.760775670 (568h51m22.631858715s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:20:22.760782372 (568h51m22.631865417s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:20:23.004266840 (568h51m22.875349895s) [seg_pos={0 0}, upload=498728]: Begin upload segment...
Dec 29 08:20:23.004343656 (568h51m22.875426701s) [seg_pos={0 0}, upload=498728]: Done begin upload segment.
Dec 29 08:20:23.004402989 (568h51m22.875486044s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.004471198 (568h51m22.875554244s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005433692 (568h51m22.876516737s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005984681 (568h51m22.877067726s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007095807 (568h51m22.878178852s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007253186 (568h51m22.878336221s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.008748694 (568h51m22.879831739s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009486750 (568h51m22.880569795s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009772374 (568h51m22.880855419s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.010493155 (568h51m22.8815762s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011140649 (568h51m22.882223694s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011589233 (568h51m22.882672268s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.012028108 (568h51m22.883111163s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014337595 (568h51m22.88542063s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014850152 (568h51m22.885933197s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.015257748 (568h51m22.886340793s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:24.618931135 (568h51m24.49001417s) [upload=498728]: Next returned nil segment
Dec 29 08:20:24.618949691 (568h51m24.490032736s) [upload=498728]: Waiting for error group managing segments...
Dec 29 08:20:25.000682572 (568h51m24.871765607s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001062866 (568h51m24.872145921s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001363920 (568h51m24.872446965s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.002695034 (568h51m24.873778079s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004076094 (568h51m24.875159129s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004874123 (568h51m24.875957158s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.019135710 (568h51m24.890218745s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.021700393 (568h51m24.892783428s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.024934521 (568h51m24.896017566s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.284220667 (568h51m25.155303712s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.310715378 (568h51m25.181798413s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.312813422 (568h51m25.183896468s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314297409 (568h51m25.185380444s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314307217 (568h51m25.185390252s) [seg_pos={0 0}, upload=498728]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:20:25.314502258 (568h51m25.185585293s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314510083 (568h51m25.185593119s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314738728 (568h51m25.185821763s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314745050 (568h51m25.185828095s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314912980 (568h51m25.185996015s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: read tcp 188.166.225.170:40388->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 29 08:20:25.314918260 (568h51m25.186001295s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314931205 (568h51m25.18601424s) [seg_pos={0 0}, upload=498728]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:20:25.314994957 (568h51m25.186077992s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:20:25.315000467 (568h51m25.186083512s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:20:25.600705118 (568h51m25.471788153s) [upload=498728]: Done uploading segments...
Dec 29 08:20:25.600725036 (568h51m25.471808071s) [upload=498728]: Upload finished. err: <nil>
Duration: 6.035307078s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [.......X........................X..X......X..........X.X..X.]
duration: 
3.404346165s
trace: 
https://jaeger.dev.storj.io/trace/7072b84d0cb8ae76
FAIL (2025-12-29 08:21:17 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.404392853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [...X.XXXX.X........X...X............XX....X.X..X...X..X...X.]
duration: 
5.700514372s
trace: 
https://jaeger.dev.storj.io/trace/65ef3e3f946e21d4
FAIL (2025-12-29 08:20:58 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA) context deadline exceeded; piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ) context deadline exceeded
======LOGS======
Dec 29 08:20:22.760401777 (568h51m22.631484823s) [upload=498728]: Starting upload
Dec 29 08:20:22.760484224 (568h51m22.631567269s) [upload=498728]: Done starting upload
Dec 29 08:20:22.760570930 (568h51m22.631653975s) [upload=498728]: Uploading segments...
Dec 29 08:20:22.760731294 (568h51m22.63181434s) [upload=498728]: Got next segment. Inline: false
Dec 29 08:20:22.760775670 (568h51m22.631858715s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:20:22.760782372 (568h51m22.631865417s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:20:23.004266840 (568h51m22.875349895s) [seg_pos={0 0}, upload=498728]: Begin upload segment...
Dec 29 08:20:23.004343656 (568h51m22.875426701s) [seg_pos={0 0}, upload=498728]: Done begin upload segment.
Dec 29 08:20:23.004402989 (568h51m22.875486044s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.004471198 (568h51m22.875554244s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005433692 (568h51m22.876516737s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005984681 (568h51m22.877067726s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007095807 (568h51m22.878178852s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007253186 (568h51m22.878336221s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.008748694 (568h51m22.879831739s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009486750 (568h51m22.880569795s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009772374 (568h51m22.880855419s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.010493155 (568h51m22.8815762s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011140649 (568h51m22.882223694s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011589233 (568h51m22.882672268s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.012028108 (568h51m22.883111163s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014337595 (568h51m22.88542063s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014850152 (568h51m22.885933197s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.015257748 (568h51m22.886340793s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:24.618931135 (568h51m24.49001417s) [upload=498728]: Next returned nil segment
Dec 29 08:20:24.618949691 (568h51m24.490032736s) [upload=498728]: Waiting for error group managing segments...
Dec 29 08:20:25.000682572 (568h51m24.871765607s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001062866 (568h51m24.872145921s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001363920 (568h51m24.872446965s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.002695034 (568h51m24.873778079s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004076094 (568h51m24.875159129s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004874123 (568h51m24.875957158s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.019135710 (568h51m24.890218745s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.021700393 (568h51m24.892783428s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.024934521 (568h51m24.896017566s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.284220667 (568h51m25.155303712s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.310715378 (568h51m25.181798413s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.312813422 (568h51m25.183896468s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314297409 (568h51m25.185380444s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314307217 (568h51m25.185390252s) [seg_pos={0 0}, upload=498728]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:20:25.314502258 (568h51m25.185585293s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314510083 (568h51m25.185593119s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314738728 (568h51m25.185821763s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314745050 (568h51m25.185828095s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314912980 (568h51m25.185996015s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: read tcp 188.166.225.170:40388->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 29 08:20:25.314918260 (568h51m25.186001295s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314931205 (568h51m25.18601424s) [seg_pos={0 0}, upload=498728]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:20:25.314994957 (568h51m25.186077992s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:20:25.315000467 (568h51m25.186083512s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:20:25.600705118 (568h51m25.471788153s) [upload=498728]: Done uploading segments...
Dec 29 08:20:25.600725036 (568h51m25.471808071s) [upload=498728]: Upload finished. err: <nil>
Duration: 5.700529322s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
17 out of 60 is failed (threshold=3) [X...X.XXXX.X........X...X............XX....X.X..X...X..X...X]
duration: 
30.002320779s
trace: 
https://jaeger.dev.storj.io/trace/65ef3e3f946e21d4
FAIL (2025-12-29 08:20:52 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA) context deadline exceeded; piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ) context deadline exceeded; piecestore: (Node ID: 1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, Piece ID: L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ) context deadline exceeded
======LOGS======
Dec 29 08:20:22.760401777 (568h51m22.631484823s) [upload=498728]: Starting upload
Dec 29 08:20:22.760484224 (568h51m22.631567269s) [upload=498728]: Done starting upload
Dec 29 08:20:22.760570930 (568h51m22.631653975s) [upload=498728]: Uploading segments...
Dec 29 08:20:22.760731294 (568h51m22.63181434s) [upload=498728]: Got next segment. Inline: false
Dec 29 08:20:22.760775670 (568h51m22.631858715s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:20:22.760782372 (568h51m22.631865417s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:20:23.004266840 (568h51m22.875349895s) [seg_pos={0 0}, upload=498728]: Begin upload segment...
Dec 29 08:20:23.004343656 (568h51m22.875426701s) [seg_pos={0 0}, upload=498728]: Done begin upload segment.
Dec 29 08:20:23.004402989 (568h51m22.875486044s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.004471198 (568h51m22.875554244s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005433692 (568h51m22.876516737s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.005984681 (568h51m22.877067726s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007095807 (568h51m22.878178852s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.007253186 (568h51m22.878336221s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.008748694 (568h51m22.879831739s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009486750 (568h51m22.880569795s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.009772374 (568h51m22.880855419s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.010493155 (568h51m22.8815762s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011140649 (568h51m22.882223694s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.011589233 (568h51m22.882672268s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.012028108 (568h51m22.883111163s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014337595 (568h51m22.88542063s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.014850152 (568h51m22.885933197s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:23.015257748 (568h51m22.886340793s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Uploading piece...
Dec 29 08:20:24.618931135 (568h51m24.49001417s) [upload=498728]: Next returned nil segment
Dec 29 08:20:24.618949691 (568h51m24.490032736s) [upload=498728]: Waiting for error group managing segments...
Dec 29 08:20:25.000682572 (568h51m24.871765607s) [noise=true , address=145.79.227.15:4876   , piece_id=FFSJ5KUXQUGY6AJKYKYW6BWIXW7PNAJXW4Z4FZTZ7YUHM5U4HUTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001062866 (568h51m24.872145921s) [noise=true , address=145.79.227.50:4373   , piece_id=OHTZEOZNJYV6ZRUPAHIFQKIQPTIMVB2VH5KZ5HT5OUE3L5J7KYUQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.001363920 (568h51m24.872446965s) [noise=true , address=145.79.227.14:4839   , piece_id=VFSYILVYOHLBTYOPHYF2332V63CNDCDL6MXI7745SILJHYVYIMWA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.002695034 (568h51m24.873778079s) [noise=true , address=145.79.227.4:4186    , piece_id=F5YURRUY2FNROG6Q7JDEZ4NACVVKHRWHEA7PNOZUVHZ6RZ2U6M5Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004076094 (568h51m24.875159129s) [noise=true , address=145.79.227.47:4266   , piece_id=E5OQXGYGW6V7URHOK2EYBJO3L2TJJE4D5NJTHEEJC74MYVVRTO4A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.004874123 (568h51m24.875957158s) [noise=true , address=145.79.227.25:4618   , piece_id=7EWNQPH225MSC64TJNQC63MWS7672ROKEVWWN2EFURP4MV5OVNQA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.019135710 (568h51m24.890218745s) [noise=true , address=145.79.227.48:4303   , piece_id=PWFKFMNDFJMLR5KWCNRIQUS6RRZKVLWB3ZWK5N3Y7SMCETAQH2IA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.021700393 (568h51m24.892783428s) [noise=true , address=145.79.227.11:4731   , piece_id=THTVNYN2XANKDW5DD7AMWK7VWGDDWWBVYOJO2ALPCBTO74GKPSEQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.024934521 (568h51m24.896017566s) [noise=true , address=145.79.227.37:4566   , piece_id=WH75K4JDP5GJL2RA27HCIWPWYX3ZUHQCSRJPDAFKFKKD2VRIHZ6Q, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.284220667 (568h51m25.155303712s) [noise=true , address=77.90.31.19:4841     , piece_id=WERHE7QDYXVKW4MPAYTN6BRZG6K3UIF6B46E5IBJFGZSN3UVOUKQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.310715378 (568h51m25.181798413s) [noise=true , address=45.91.116.11:4119    , piece_id=VLVUZ5B7JBYWHAAPMR2WPBIEXLJ466NYVS3DNB4QQCNBTTA6IZ6A, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.312813422 (568h51m25.183896468s) [noise=true , address=45.91.116.13:4155    , piece_id=K3GY7NF3VMZSKBHIEVIF5A24QGBTZ2WDFKP6F66YOVRL4U7YC4XA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314297409 (568h51m25.185380444s) [noise=true , address=45.91.116.7:4010     , piece_id=L73JIK6YHSHWYHISNMAZNYXHBLET6ZB3POCKIKEPFTBMPTHREBTQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: <nil>
Dec 29 08:20:25.314307217 (568h51m25.185390252s) [seg_pos={0 0}, upload=498728]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:20:25.314502258 (568h51m25.185585293s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314510083 (568h51m25.185593119s) [noise=true , address=77.90.31.11:4554     , piece_id=6TK266OMSQHCXF3YYTM7WIGJCHULS7OYPSVNDQTMLXJQ6EAKIWFA, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314738728 (568h51m25.185821763s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF
Dec 29 08:20:25.314745050 (568h51m25.185828095s) [noise=true , address=45.91.116.4:4940     , piece_id=RNQ6PW3JNDKI63HVU3XCL7X237OYSCQ3UPYDOYTIRORXU2R5M4PQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314912980 (568h51m25.185996015s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: read tcp 188.166.225.170:40388->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 29 08:20:25.314918260 (568h51m25.186001295s) [noise=true , address=45.91.116.12:4228    , piece_id=PWPZYSJLDECMHZA52BEX5MSLUDOWZYIHZFKYDPOFMMHSCNY5R6TQ, seg_pos={0 0}, upload=498728]: Long tail cancellation due to optimal threshold
Dec 29 08:20:25.314931205 (568h51m25.18601424s) [seg_pos={0 0}, upload=498728]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:20:25.314994957 (568h51m25.186077992s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:20:25.315000467 (568h51m25.186083512s) [upload=498728]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:20:25.600705118 (568h51m25.471788153s) [upload=498728]: Done uploading segments...
Dec 29 08:20:25.600725036 (568h51m25.471808071s) [upload=498728]: Upload finished. err: <nil>
Duration: 30.002343201s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [......X.X.X.............X..........................X........]
duration: 
15.572749537s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:20:32 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 15.572766559s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [........X........................X..X......X..........X.X..X]
duration: 
5.001598593s
trace: 
https://jaeger.dev.storj.io/trace/7072b84d0cb8ae76
FAIL (2025-12-29 08:20:22 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001623141s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.......X.X.X.............X..........................X.......]
duration: 
34.574881373s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:20:16 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 34.574910568s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
6 out of 60 is failed (threshold=6) [.........X........................X..X......X..........X.X..]
duration: 
3.512827541s
trace: 
https://jaeger.dev.storj.io/trace/47f89b816a5def61
FAIL (2025-12-29 08:20:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.512857649s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [.X...X.XXXX.X........X...X............XX....X.X..X...X..X...]
duration: 
5.976380161s
trace: 
https://jaeger.dev.storj.io/trace/4253e4135707e926
FAIL (2025-12-29 08:19:40 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: 6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ) context deadline exceeded
======LOGS======
ng segments...
Dec 29 08:17:34.112590969 (568h48m33.983674014s) [upload=498684]: Got next segment. Inline: false
Dec 29 08:17:34.112653928 (568h48m33.983736973s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:17:34.112660741 (568h48m33.983743776s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:17:34.359695048 (568h48m34.230778083s) [seg_pos={0 0}, upload=498684]: Begin upload segment...
Dec 29 08:17:34.359736016 (568h48m34.230819051s) [seg_pos={0 0}, upload=498684]: Done begin upload segment.
Dec 29 08:17:34.359761985 (568h48m34.23084502s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.359836818 (568h48m34.230919853s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360686867 (568h48m34.231769902s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360753703 (568h48m34.231836748s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361153205 (568h48m34.23223625s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361451672 (568h48m34.232534707s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361691790 (568h48m34.232774825s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361917390 (568h48m34.233000425s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362309246 (568h48m34.233392291s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362757620 (568h48m34.233840655s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362849705 (568h48m34.23393274s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363069443 (568h48m34.234152478s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363343455 (568h48m34.23442649s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363379824 (568h48m34.23446289s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364072995 (568h48m34.23515604s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364643310 (568h48m34.235726345s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:36.052244115 (568h48m35.92332715s) [upload=498684]: Next returned nil segment
Dec 29 08:17:36.052258543 (568h48m35.923341578s) [upload=498684]: Waiting for error group managing segments...
Dec 29 08:17:36.348726619 (568h48m36.219809654s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350071691 (568h48m36.221154726s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350601259 (568h48m36.221684294s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.353158999 (568h48m36.224242034s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.360612271 (568h48m36.231695306s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.361757382 (568h48m36.232840417s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.366672631 (568h48m36.237755676s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.369475427 (568h48m36.240558472s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375104046 (568h48m36.246187091s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375785373 (568h48m36.246868408s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.544413915 (568h48m36.41549695s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.585013482 (568h48m36.456096527s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604180228 (568h48m36.475263263s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604207710 (568h48m36.475290755s) [seg_pos={0 0}, upload=498684]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:17:36.604439812 (568h48m36.475522857s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection
Dec 29 08:17:36.604449020 (568h48m36.475532065s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604579458 (568h48m36.475662493s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection
Dec 29 08:17:36.604615387 (568h48m36.475698432s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604780240 (568h48m36.475863285s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF
Dec 29 08:17:36.604791131 (568h48m36.475874166s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604815156 (568h48m36.475898191s) [seg_pos={0 0}, upload=498684]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:17:36.604896812 (568h48m36.475979857s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:17:36.604905008 (568h48m36.475988073s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:17:36.901856736 (568h48m36.772939781s) [upload=498684]: Done uploading segments...
Dec 29 08:17:36.901879739 (568h48m36.772962774s) [upload=498684]: Upload finished. err: <nil>
Duration: 5.976408776s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [........X.X.X.............X..........................X......]
duration: 
30.159975802s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:19:16 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 30.16001056s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
6 out of 60 is failed (threshold=6) [..........X........................X..X......X..........X.X.]
duration: 
3.518592667s
trace: 
https://jaeger.dev.storj.io/trace/47f89b816a5def61
FAIL (2025-12-29 08:19:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.518615491s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
16 out of 60 is failed (threshold=3) [..X...X.XXXX.X........X...X............XX....X.X..X...X..X..]
duration: 
5.842413531s
trace: 
https://jaeger.dev.storj.io/trace/4253e4135707e926
FAIL (2025-12-29 08:18:55 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: 6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ) context deadline exceeded
======LOGS======
ng segments...
Dec 29 08:17:34.112590969 (568h48m33.983674014s) [upload=498684]: Got next segment. Inline: false
Dec 29 08:17:34.112653928 (568h48m33.983736973s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:17:34.112660741 (568h48m33.983743776s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:17:34.359695048 (568h48m34.230778083s) [seg_pos={0 0}, upload=498684]: Begin upload segment...
Dec 29 08:17:34.359736016 (568h48m34.230819051s) [seg_pos={0 0}, upload=498684]: Done begin upload segment.
Dec 29 08:17:34.359761985 (568h48m34.23084502s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.359836818 (568h48m34.230919853s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360686867 (568h48m34.231769902s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360753703 (568h48m34.231836748s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361153205 (568h48m34.23223625s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361451672 (568h48m34.232534707s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361691790 (568h48m34.232774825s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361917390 (568h48m34.233000425s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362309246 (568h48m34.233392291s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362757620 (568h48m34.233840655s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362849705 (568h48m34.23393274s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363069443 (568h48m34.234152478s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363343455 (568h48m34.23442649s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363379824 (568h48m34.23446289s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364072995 (568h48m34.23515604s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364643310 (568h48m34.235726345s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:36.052244115 (568h48m35.92332715s) [upload=498684]: Next returned nil segment
Dec 29 08:17:36.052258543 (568h48m35.923341578s) [upload=498684]: Waiting for error group managing segments...
Dec 29 08:17:36.348726619 (568h48m36.219809654s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350071691 (568h48m36.221154726s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350601259 (568h48m36.221684294s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.353158999 (568h48m36.224242034s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.360612271 (568h48m36.231695306s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.361757382 (568h48m36.232840417s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.366672631 (568h48m36.237755676s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.369475427 (568h48m36.240558472s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375104046 (568h48m36.246187091s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375785373 (568h48m36.246868408s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.544413915 (568h48m36.41549695s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.585013482 (568h48m36.456096527s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604180228 (568h48m36.475263263s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604207710 (568h48m36.475290755s) [seg_pos={0 0}, upload=498684]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:17:36.604439812 (568h48m36.475522857s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection
Dec 29 08:17:36.604449020 (568h48m36.475532065s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604579458 (568h48m36.475662493s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection
Dec 29 08:17:36.604615387 (568h48m36.475698432s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604780240 (568h48m36.475863285s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF
Dec 29 08:17:36.604791131 (568h48m36.475874166s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604815156 (568h48m36.475898191s) [seg_pos={0 0}, upload=498684]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:17:36.604896812 (568h48m36.475979857s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:17:36.604905008 (568h48m36.475988073s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:17:36.901856736 (568h48m36.772939781s) [upload=498684]: Done uploading segments...
Dec 29 08:17:36.901879739 (568h48m36.772962774s) [upload=498684]: Upload finished. err: <nil>
Duration: 5.842440522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
5 out of 60 is failed (threshold=3) [.........X.X.X.............X..........................X.....]
duration: 
25.013981938s
trace: 
https://jaeger.dev.storj.io/trace/a6340b01349e0d
FAIL (2025-12-29 08:18:46 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.247819992 (568h45m36.118903027s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248342738 (568h45m36.119425793s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.248847809 (568h45m36.119930944s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249356217 (568h45m36.120439272s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.249944006 (568h45m36.121027041s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.250484676 (568h45m36.121567731s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.252307867 (568h45m36.123390912s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.254200411 (568h45m36.125283456s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.256317642 (568h45m36.127400707s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258310357 (568h45m36.129393412s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.258396260 (568h45m36.129479306s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.259535260 (568h45m36.130618305s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.260603623 (568h45m36.131686658s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261823867 (568h45m36.132906902s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:36.261964755 (568h45m36.1330478s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Uploading piece...
Dec 29 08:14:37.701154511 (568h45m37.572237546s) [upload=498639]: Next returned nil segment
Dec 29 08:14:37.701167917 (568h45m37.572250962s) [upload=498639]: Waiting for error group managing segments...
Dec 29 08:14:38.099583464 (568h45m37.9706665s) [noise=true , address=145.79.227.17:4988   , piece_id=FTQAV3Z2SKB4OQXK3AR47Z3POPO4WHKQQTUL3RL7M3W6ZYNVN2KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.107512071 (568h45m37.978595106s) [noise=true , address=145.79.227.19:4091   , piece_id=N7ZJCMJQO7N6WLQ6G333XFU5OGMJL2THXXC6MOLBOFZGXCLQXWQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.118701377 (568h45m37.989784412s) [noise=true , address=64.49.13.140:4513    , piece_id=YCCKSXVZVIYUWVLCVMFJP5M5AHXRNKMMRELPEXI6DYSBOIZYDDHA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.121990390 (568h45m37.993073436s) [noise=true , address=145.79.227.6:4275    , piece_id=HCFQBSFOAHWUFX577XBGRGXEM5BA2G4LUZKTNXAGDZS6QE2NC6KA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.130817759 (568h45m38.001900804s) [noise=true , address=145.79.227.29:4946   , piece_id=KOGEA4IIZVDMURP4XUX4XTZLP7FFIFVUSCGOG3O6DSML6JTESZOQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.140792001 (568h45m38.011875036s) [noise=true , address=145.79.227.7:4302    , piece_id=672P4QLMXU4J6AYBMG2F5TID3T3PBDZ5KHSFTWVSIUSWKQ2ALQDQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.308459122 (568h45m38.179542168s) [noise=true , address=77.90.31.14:4665     , piece_id=5TVEIZ6ND2NCI4DZKVTBC5J7WOLHTYZTTCOFH3BPZGUFRHMVL3OQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.352666790 (568h45m38.223749825s) [noise=true , address=77.90.31.18:4832     , piece_id=LJBNOAJWM3CBF6URNX3AUBBNWVC3RWFGTVIU5CWS5GI5R6VFECOA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.448694279 (568h45m38.319777324s) [noise=true , address=77.90.31.15:4724     , piece_id=6FFXZPBGIFVPSU3B5722L7TD4K6JJERWLGJND5PAMAJAIZQMGSSQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.455222761 (568h45m38.326305806s) [noise=true , address=45.91.116.14:4207    , piece_id=6UG4LG76D7DLBPXDGMHW7RLXYBJQXJUHJIGE7RWGB3FPUYEHPJNQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.496227930 (568h45m38.367310975s) [noise=true , address=45.91.116.6:4986     , piece_id=Z7EAQNBEEAD7PG64B72GNJ55JT2GXTCD3MA2WZDS5QJQVQTGRQQQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:38.975868381 (568h45m38.846951417s) [noise=true , address=145.79.227.34:4930   , piece_id=KS6U2OTGZ5L3CZ2M6HC4GZZEJYSRZCB75SMKLVVCWQLIBBV7AZYQ, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.027529080 (568h45m38.898612125s) [noise=true , address=145.79.227.30:4772   , piece_id=F4M7OXPFSOGSOJTVYJ5TVRCNSMK2JA7HLP3XMVCTY5QM52RGIN6Q, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059807343 (568h45m38.930890389s) [noise=true , address=145.79.227.44:4201   , piece_id=ZA2P5IKR7GYGFJYOBWMLE6CRCEUFCTANYCLX7QSUAHEVZEYHWHLA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: <nil>
Dec 29 08:14:39.059832932 (568h45m38.930915967s) [seg_pos={0 0}, upload=498639]: Segment reached optimal threshold of 14 pieces.
Dec 29 08:14:39.060092186 (568h45m38.931175241s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060105542 (568h45m38.931188587s) [noise=true , address=77.90.31.13:4645     , piece_id=FFHY4TLTCB6HF4PDYGRIRCJLTHHCZLBUJSGB7L7T4H67GK7J6EMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060153042 (568h45m38.931236077s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qFj8Hrgxx3ZMjo1tzyJ5nqjGmF3ahUuisy2rXWLztxDRVRR4K): protocol: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40112->64.49.13.169:4709: use of closed network connection
Dec 29 08:14:39.060170705 (568h45m38.93125375s) [noise=true , address=64.49.13.169:4709    , piece_id=TTYPDIIT662HPX3XANNYFQFR2XV63NE4OS2HZMUYF7BNLG2LFA6A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060407406 (568h45m38.931490441s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060413978 (568h45m38.931497023s) [noise=true , address=145.79.227.33:4884   , piece_id=TF7KDEVRAL54N7ZMQ32B7IB5CY6LDRPNN72DMSWXVO3N2ELU622A, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060447702 (568h45m38.931530747s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE): protocol: expected piece hash; context canceled; EOF
Dec 29 08:14:39.060456971 (568h45m38.931540016s) [noise=true , address=77.90.31.16:4750     , piece_id=GMPTFGSLUU4G65ASVJL5B7S3WRCYJRJWCNBQTC2APVLEIGRD2PMA, seg_pos={0 0}, upload=498639]: Long tail cancellation due to optimal threshold
Dec 29 08:14:39.060490455 (568h45m38.9315735s) [seg_pos={0 0}, upload=498639]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 29 08:14:39.060588581 (568h45m38.931671616s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:14:39.060594803 (568h45m38.931677848s) [upload=498639]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:14:39.360082939 (568h45m39.231165974s) [upload=498639]: Done uploading segments...
Dec 29 08:14:39.360106114 (568h45m39.231189159s) [upload=498639]: Upload finished. err: <nil>
Duration: 25.013999441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
17 out of 60 is failed (threshold=3) [X..X...X.XXXX.X........X...X............XX....X.X..X...X..X.]
duration: 
28.913909705s
trace: 
https://jaeger.dev.storj.io/trace/4253e4135707e926
FAIL (2025-12-29 08:18:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA) context deadline exceeded; piecestore: (Node ID: 1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, Piece ID: 6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ) context deadline exceeded
======LOGS======
ng segments...
Dec 29 08:17:34.112590969 (568h48m33.983674014s) [upload=498684]: Got next segment. Inline: false
Dec 29 08:17:34.112653928 (568h48m33.983736973s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 29 08:17:34.112660741 (568h48m33.983743776s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 29 08:17:34.359695048 (568h48m34.230778083s) [seg_pos={0 0}, upload=498684]: Begin upload segment...
Dec 29 08:17:34.359736016 (568h48m34.230819051s) [seg_pos={0 0}, upload=498684]: Done begin upload segment.
Dec 29 08:17:34.359761985 (568h48m34.23084502s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.359836818 (568h48m34.230919853s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360686867 (568h48m34.231769902s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.360753703 (568h48m34.231836748s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361153205 (568h48m34.23223625s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361451672 (568h48m34.232534707s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361691790 (568h48m34.232774825s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.361917390 (568h48m34.233000425s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362309246 (568h48m34.233392291s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362757620 (568h48m34.233840655s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.362849705 (568h48m34.23393274s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363069443 (568h48m34.234152478s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363343455 (568h48m34.23442649s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.363379824 (568h48m34.23446289s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364072995 (568h48m34.23515604s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:34.364643310 (568h48m34.235726345s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Uploading piece...
Dec 29 08:17:36.052244115 (568h48m35.92332715s) [upload=498684]: Next returned nil segment
Dec 29 08:17:36.052258543 (568h48m35.923341578s) [upload=498684]: Waiting for error group managing segments...
Dec 29 08:17:36.348726619 (568h48m36.219809654s) [noise=true , address=145.79.227.45:4229   , piece_id=I7FHQUDAGZRUCEKQMEEHU6IDGB46T4X6G3XJOVPXEO5DK7TEV7WA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350071691 (568h48m36.221154726s) [noise=true , address=145.79.227.12:4767   , piece_id=YISVXWRH7JAUHT5CXQDXEKGEL4YKR3V5UWHKV2ZZNFYMUQHZ2WBA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.350601259 (568h48m36.221684294s) [noise=true , address=145.79.227.42:4122   , piece_id=S2HCS4J2OIALSQFYIEWI54HPGNAJQA5ZG5Y33X3JR6HQNVWKLTGQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.353158999 (568h48m36.224242034s) [noise=true , address=145.79.227.46:4601   , piece_id=RAEG4HD5ULQUW2QUJVJ5D6U4OZ23JJHYXMXNG2RSI5FM6X5OQWTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.360612271 (568h48m36.231695306s) [noise=true , address=145.79.227.5:4221    , piece_id=6T7HIJFKLR43NV2G2XS23NJLY33D3YJXOAP3NQXMZUPCH6KWPEXA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.361757382 (568h48m36.232840417s) [noise=true , address=145.79.227.51:4410   , piece_id=XHPO42W4GQ2JWWJJXZPDIQJGRIRPBSFG5IDSNHJ2746B4JUCOBTA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.366672631 (568h48m36.237755676s) [noise=true , address=145.79.227.50:4375   , piece_id=V5XO4KQINXMAARSOQMRLQVCS2LTZYXFU6EBMUP2OXD3TKY2MQV4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.369475427 (568h48m36.240558472s) [noise=true , address=145.79.227.36:4518   , piece_id=TP4QM427VOYNUH65RYPHW5P56YSK2FH52GHXWM66JIGMJ5JDGY4A, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375104046 (568h48m36.246187091s) [noise=true , address=145.79.227.52:4446   , piece_id=FAXKLDWJZQ7B4LRIXHQFWAPZFRY75EYNL6IXNX5EXB5CSQCU5HZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.375785373 (568h48m36.246868408s) [noise=true , address=145.79.227.4:4185    , piece_id=LRGRJBO5MVWH6DPN5GE4U6NHDSRW6KUJYPUHMGT74M3R7XPJU7IA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.544413915 (568h48m36.41549695s) [noise=true , address=77.90.31.10:4518     , piece_id=J4HYBBZF3KH6ERB3Y6Y7CQDYIVXSP473HOXG4HSKT5OHXFNEFQZA, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.585013482 (568h48m36.456096527s) [noise=true , address=45.91.116.4:4937     , piece_id=U7OHESEC2GNESAWH3QZCXPFK4QKEQNALK4I7QJOEBWM5QJVLW27Q, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604180228 (568h48m36.475263263s) [noise=true , address=77.90.31.11:4556     , piece_id=6Z7BPLVLW4OBHCYXJFDKLYHXUI4KPEK2FCHVAD5Z4K74UVDI2GUQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: <nil>
Dec 29 08:17:36.604207710 (568h48m36.475290755s) [seg_pos={0 0}, upload=498684]: Segment reached optimal threshold of 13 pieces.
Dec 29 08:17:36.604439812 (568h48m36.475522857s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42840->77.90.31.21:4878: use of closed network connection
Dec 29 08:17:36.604449020 (568h48m36.475532065s) [noise=true , address=77.90.31.21:4878     , piece_id=NKQH2KZOLYR2JJMG7DVO64MNLLQS6C3H2XJVOWFRIZ7LVXBXJJLQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604579458 (568h48m36.475662493s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58618->45.91.116.8:4046: use of closed network connection
Dec 29 08:17:36.604615387 (568h48m36.475698432s) [noise=true , address=45.91.116.8:4046     , piece_id=5AZGS4N5C34QKG6HLT6FKGZA7B526MVOWRIWQASN2SXUO5FLGSKQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604780240 (568h48m36.475863285s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF
Dec 29 08:17:36.604791131 (568h48m36.475874166s) [noise=true , address=45.91.116.9:4262     , piece_id=7JM55ZQLHKD6E4EV7LUWWAWXCJIC6WKYRWO7UG26H7SXVYIXF7VQ, seg_pos={0 0}, upload=498684]: Long tail cancellation due to optimal threshold
Dec 29 08:17:36.604815156 (568h48m36.475898191s) [seg_pos={0 0}, upload=498684]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 29 08:17:36.604896812 (568h48m36.475979857s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 29 08:17:36.604905008 (568h48m36.475988073s) [upload=498684]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 29 08:17:36.901856736 (568h48m36.772939781s) [upload=498684]: Done uploading segments...
Dec 29 08:17:36.901879739 (568h48m36.772962774s) [upload=498684]: Upload finished. err: <nil>
Duration: 28.913933249s core uplink us-select-1