Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 354154 reports (Page 211 of 7084)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [....................X.....................X.................]
duration: 
39.334532912s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:04:17 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 39.334558301s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [......................X....X.....X.........X...X.X....X.X...]
duration: 
5.001083808s
trace: 
https://jaeger.dev.storj.io/trace/41fd4dc7b7a07666
FAIL (2025-12-30 09:04:05 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001113313s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [.......................X....X.....X.........X...X.X....X.X..]
duration: 
3.713359756s
trace: 
https://jaeger.dev.storj.io/trace/41fd4dc7b7a07666
FAIL (2025-12-30 09:03:33 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.713379983s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
19 out of 60 is failed (threshold=3) [.X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X....X.]
duration: 
5.947967598s
trace: 
https://jaeger.dev.storj.io/trace/747e3cb14f34091a
FAIL (2025-12-30 09:03:23 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA) context deadline exceeded
======LOGS======
ments...
Dec 30 09:02:31.667859845 (593h33m31.53894289s) [upload=520344]: Got next segment. Inline: false
Dec 30 09:02:31.667898849 (593h33m31.538981894s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:02:31.667906814 (593h33m31.538989849s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:02:31.911288441 (593h33m31.782371486s) [seg_pos={0 0}, upload=520344]: Begin upload segment...
Dec 30 09:02:31.911346743 (593h33m31.782429788s) [seg_pos={0 0}, upload=520344]: Done begin upload segment.
Dec 30 09:02:31.911383002 (593h33m31.782466047s) [noise=true , address=145.79.227.7:4293    , piece_id=RDEJMGXX72HIIU6JALZACV5KA4VZ3I4TAKAOU6XT7BSEEIRWZCUQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.911411536 (593h33m31.782494571s) [noise=true , address=145.79.227.29:4942   , piece_id=7AVNZKRYFLGL5V33ZLXKBVYWYQXQN5FEU5OOYNEER2AC3DZSEW7Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.911588753 (593h33m31.782671809s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.912264780 (593h33m31.783347825s) [noise=true , address=145.79.227.23:4546   , piece_id=FEPBCKM43NMMSCCAZWJE6IVJU4KWVLY2DULMBE2F7YEKTJLM7XZA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.912781033 (593h33m31.783864079s) [noise=true , address=145.79.227.18:4006   , piece_id=NPYRAQU7BPTAJPH2FBLACX5O4J36GJCETAH4Z6YU6WHRICOGRKVQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913222294 (593h33m31.784305329s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913229317 (593h33m31.784312352s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913694243 (593h33m31.784777278s) [noise=true , address=145.79.227.13:4803   , piece_id=VALIJXNJMELYU72H3L6WL2J3OBSFA2XYKQV7I6ALCVOAWPH3KIRA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914155191 (593h33m31.785238236s) [noise=true , address=77.90.31.11:4553     , piece_id=FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914429273 (593h33m31.785512318s) [noise=true , address=145.79.227.19:4079   , piece_id=63EP2O52E2RY4KJJRTA4LFIVFLD5QYGQKHAOKUJKHLMTYFYKRR3A, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914615036 (593h33m31.785698081s) [noise=true , address=77.90.31.14:4661     , piece_id=VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914938672 (593h33m31.786021707s) [noise=true , address=145.79.227.14:4837   , piece_id=44FB5TAUY67D4OSR7SK3RPS7NZGEVDXM44WHMYGTK3WMHLEVOOWA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.915074130 (593h33m31.786157175s) [noise=true , address=145.79.227.8:4331    , piece_id=UFY5OCZIYHFBC7RYMO22FNICZIFC7UFETQF45DULRCJF5KUYXSVA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.915549806 (593h33m31.786632841s) [noise=true , address=145.79.227.11:4730   , piece_id=KNNS7BDD4SCKWXY3KBY7DCGDMBWNFJBOBDN5ZSKXFYFZXB6SN24A, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.916085726 (593h33m31.787168781s) [noise=true , address=145.79.227.15:4873   , piece_id=T2XAEMNYTOLPCCHTGK3UXQY65TMSSMCM3NINPZGQENVZ4OPLQ4SA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.918195133 (593h33m31.789278168s) [noise=true , address=145.79.227.12:4765   , piece_id=3MN4RNU4D274ADTOCJJ4ITWCFWUBLEGMAD5Q7VMEJ5SDQIE6IKCQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:33.515920470 (593h33m33.387003515s) [upload=520344]: Next returned nil segment
Dec 30 09:02:33.515935207 (593h33m33.387018262s) [upload=520344]: Waiting for error group managing segments...
Dec 30 09:02:33.898381507 (593h33m33.769464552s) [noise=true , address=145.79.227.18:4006   , piece_id=NPYRAQU7BPTAJPH2FBLACX5O4J36GJCETAH4Z6YU6WHRICOGRKVQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.898450178 (593h33m33.769533223s) [noise=true , address=145.79.227.29:4942   , piece_id=7AVNZKRYFLGL5V33ZLXKBVYWYQXQN5FEU5OOYNEER2AC3DZSEW7Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.900956380 (593h33m33.772039425s) [noise=true , address=145.79.227.11:4730   , piece_id=KNNS7BDD4SCKWXY3KBY7DCGDMBWNFJBOBDN5ZSKXFYFZXB6SN24A, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.902207612 (593h33m33.773290657s) [noise=true , address=145.79.227.12:4765   , piece_id=3MN4RNU4D274ADTOCJJ4ITWCFWUBLEGMAD5Q7VMEJ5SDQIE6IKCQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.903765930 (593h33m33.774848965s) [noise=true , address=145.79.227.19:4079   , piece_id=63EP2O52E2RY4KJJRTA4LFIVFLD5QYGQKHAOKUJKHLMTYFYKRR3A, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.904393485 (593h33m33.77547652s) [noise=true , address=145.79.227.15:4873   , piece_id=T2XAEMNYTOLPCCHTGK3UXQY65TMSSMCM3NINPZGQENVZ4OPLQ4SA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.912669343 (593h33m33.783752378s) [noise=true , address=145.79.227.23:4546   , piece_id=FEPBCKM43NMMSCCAZWJE6IVJU4KWVLY2DULMBE2F7YEKTJLM7XZA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.927765218 (593h33m33.798848253s) [noise=true , address=145.79.227.7:4293    , piece_id=RDEJMGXX72HIIU6JALZACV5KA4VZ3I4TAKAOU6XT7BSEEIRWZCUQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.943681546 (593h33m33.814764591s) [noise=true , address=145.79.227.8:4331    , piece_id=UFY5OCZIYHFBC7RYMO22FNICZIFC7UFETQF45DULRCJF5KUYXSVA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.987038454 (593h33m33.858121499s) [noise=true , address=145.79.227.14:4837   , piece_id=44FB5TAUY67D4OSR7SK3RPS7NZGEVDXM44WHMYGTK3WMHLEVOOWA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.992443416 (593h33m33.863526451s) [noise=true , address=145.79.227.13:4803   , piece_id=VALIJXNJMELYU72H3L6WL2J3OBSFA2XYKQV7I6ALCVOAWPH3KIRA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.089020444 (593h33m33.960103489s) [noise=true , address=77.90.31.11:4553     , piece_id=FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.231364450 (593h33m34.102447495s) [noise=true , address=77.90.31.14:4661     , piece_id=VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.231391521 (593h33m34.102474566s) [seg_pos={0 0}, upload=520344]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:02:34.231603606 (593h33m34.102686651s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection
Dec 30 09:02:34.231680091 (593h33m34.102763136s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.231933874 (593h33m34.103016909s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection
Dec 30 09:02:34.231954163 (593h33m34.103037198s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.232042731 (593h33m34.103125766s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: expected piece hash; context canceled; EOF
Dec 30 09:02:34.232052450 (593h33m34.103135495s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.232073310 (593h33m34.103156355s) [seg_pos={0 0}, upload=520344]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:02:34.232157620 (593h33m34.103240655s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:02:34.232164383 (593h33m34.103247429s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:02:34.521600667 (593h33m34.392683712s) [upload=520344]: Done uploading segments...
Dec 30 09:02:34.521623249 (593h33m34.392706294s) [upload=520344]: Upload finished. err: <nil>
Duration: 5.947994618s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.....................X.....................X................]
duration: 
28.481149426s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:03:10 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 28.481176987s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
19 out of 60 is failed (threshold=3) [..X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X....X]
duration: 
30.003957111s
trace: 
https://jaeger.dev.storj.io/trace/747e3cb14f34091a
FAIL (2025-12-30 09:03:00 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA) context deadline exceeded
======LOGS======
ments...
Dec 30 09:02:31.667859845 (593h33m31.53894289s) [upload=520344]: Got next segment. Inline: false
Dec 30 09:02:31.667898849 (593h33m31.538981894s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 09:02:31.667906814 (593h33m31.538989849s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 09:02:31.911288441 (593h33m31.782371486s) [seg_pos={0 0}, upload=520344]: Begin upload segment...
Dec 30 09:02:31.911346743 (593h33m31.782429788s) [seg_pos={0 0}, upload=520344]: Done begin upload segment.
Dec 30 09:02:31.911383002 (593h33m31.782466047s) [noise=true , address=145.79.227.7:4293    , piece_id=RDEJMGXX72HIIU6JALZACV5KA4VZ3I4TAKAOU6XT7BSEEIRWZCUQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.911411536 (593h33m31.782494571s) [noise=true , address=145.79.227.29:4942   , piece_id=7AVNZKRYFLGL5V33ZLXKBVYWYQXQN5FEU5OOYNEER2AC3DZSEW7Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.911588753 (593h33m31.782671809s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.912264780 (593h33m31.783347825s) [noise=true , address=145.79.227.23:4546   , piece_id=FEPBCKM43NMMSCCAZWJE6IVJU4KWVLY2DULMBE2F7YEKTJLM7XZA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.912781033 (593h33m31.783864079s) [noise=true , address=145.79.227.18:4006   , piece_id=NPYRAQU7BPTAJPH2FBLACX5O4J36GJCETAH4Z6YU6WHRICOGRKVQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913222294 (593h33m31.784305329s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913229317 (593h33m31.784312352s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.913694243 (593h33m31.784777278s) [noise=true , address=145.79.227.13:4803   , piece_id=VALIJXNJMELYU72H3L6WL2J3OBSFA2XYKQV7I6ALCVOAWPH3KIRA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914155191 (593h33m31.785238236s) [noise=true , address=77.90.31.11:4553     , piece_id=FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914429273 (593h33m31.785512318s) [noise=true , address=145.79.227.19:4079   , piece_id=63EP2O52E2RY4KJJRTA4LFIVFLD5QYGQKHAOKUJKHLMTYFYKRR3A, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914615036 (593h33m31.785698081s) [noise=true , address=77.90.31.14:4661     , piece_id=VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.914938672 (593h33m31.786021707s) [noise=true , address=145.79.227.14:4837   , piece_id=44FB5TAUY67D4OSR7SK3RPS7NZGEVDXM44WHMYGTK3WMHLEVOOWA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.915074130 (593h33m31.786157175s) [noise=true , address=145.79.227.8:4331    , piece_id=UFY5OCZIYHFBC7RYMO22FNICZIFC7UFETQF45DULRCJF5KUYXSVA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.915549806 (593h33m31.786632841s) [noise=true , address=145.79.227.11:4730   , piece_id=KNNS7BDD4SCKWXY3KBY7DCGDMBWNFJBOBDN5ZSKXFYFZXB6SN24A, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.916085726 (593h33m31.787168781s) [noise=true , address=145.79.227.15:4873   , piece_id=T2XAEMNYTOLPCCHTGK3UXQY65TMSSMCM3NINPZGQENVZ4OPLQ4SA, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:31.918195133 (593h33m31.789278168s) [noise=true , address=145.79.227.12:4765   , piece_id=3MN4RNU4D274ADTOCJJ4ITWCFWUBLEGMAD5Q7VMEJ5SDQIE6IKCQ, seg_pos={0 0}, upload=520344]: Uploading piece...
Dec 30 09:02:33.515920470 (593h33m33.387003515s) [upload=520344]: Next returned nil segment
Dec 30 09:02:33.515935207 (593h33m33.387018262s) [upload=520344]: Waiting for error group managing segments...
Dec 30 09:02:33.898381507 (593h33m33.769464552s) [noise=true , address=145.79.227.18:4006   , piece_id=NPYRAQU7BPTAJPH2FBLACX5O4J36GJCETAH4Z6YU6WHRICOGRKVQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.898450178 (593h33m33.769533223s) [noise=true , address=145.79.227.29:4942   , piece_id=7AVNZKRYFLGL5V33ZLXKBVYWYQXQN5FEU5OOYNEER2AC3DZSEW7Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.900956380 (593h33m33.772039425s) [noise=true , address=145.79.227.11:4730   , piece_id=KNNS7BDD4SCKWXY3KBY7DCGDMBWNFJBOBDN5ZSKXFYFZXB6SN24A, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.902207612 (593h33m33.773290657s) [noise=true , address=145.79.227.12:4765   , piece_id=3MN4RNU4D274ADTOCJJ4ITWCFWUBLEGMAD5Q7VMEJ5SDQIE6IKCQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.903765930 (593h33m33.774848965s) [noise=true , address=145.79.227.19:4079   , piece_id=63EP2O52E2RY4KJJRTA4LFIVFLD5QYGQKHAOKUJKHLMTYFYKRR3A, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.904393485 (593h33m33.77547652s) [noise=true , address=145.79.227.15:4873   , piece_id=T2XAEMNYTOLPCCHTGK3UXQY65TMSSMCM3NINPZGQENVZ4OPLQ4SA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.912669343 (593h33m33.783752378s) [noise=true , address=145.79.227.23:4546   , piece_id=FEPBCKM43NMMSCCAZWJE6IVJU4KWVLY2DULMBE2F7YEKTJLM7XZA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.927765218 (593h33m33.798848253s) [noise=true , address=145.79.227.7:4293    , piece_id=RDEJMGXX72HIIU6JALZACV5KA4VZ3I4TAKAOU6XT7BSEEIRWZCUQ, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.943681546 (593h33m33.814764591s) [noise=true , address=145.79.227.8:4331    , piece_id=UFY5OCZIYHFBC7RYMO22FNICZIFC7UFETQF45DULRCJF5KUYXSVA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.987038454 (593h33m33.858121499s) [noise=true , address=145.79.227.14:4837   , piece_id=44FB5TAUY67D4OSR7SK3RPS7NZGEVDXM44WHMYGTK3WMHLEVOOWA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:33.992443416 (593h33m33.863526451s) [noise=true , address=145.79.227.13:4803   , piece_id=VALIJXNJMELYU72H3L6WL2J3OBSFA2XYKQV7I6ALCVOAWPH3KIRA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.089020444 (593h33m33.960103489s) [noise=true , address=77.90.31.11:4553     , piece_id=FOZBBMKPVWOZA7QWY6APEQAN3BJ64K7B5KMUGFMN7LDBY6TCKV4Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.231364450 (593h33m34.102447495s) [noise=true , address=77.90.31.14:4661     , piece_id=VJRWZJ42DZNSESGOYHA6VTB3NOOAM7JSAHX6RBFOYADQKUDSXBEA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: <nil>
Dec 30 09:02:34.231391521 (593h33m34.102474566s) [seg_pos={0 0}, upload=520344]: Segment reached optimal threshold of 13 pieces.
Dec 30 09:02:34.231603606 (593h33m34.102686651s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37576->77.90.31.15:4697: use of closed network connection
Dec 30 09:02:34.231680091 (593h33m34.102763136s) [noise=true , address=77.90.31.15:4697     , piece_id=BZNND4D5IETZAITDXZDVLYOBMAJINETVYNFLASNZPB6YMTZWN7RA, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.231933874 (593h33m34.103016909s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection; write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33044->45.91.116.6:4975: use of closed network connection
Dec 30 09:02:34.231954163 (593h33m34.103037198s) [noise=true , address=45.91.116.6:4975     , piece_id=EEJLAEH6DPGCB5NTXDU63PRHELVGHUYP7O6GFMTBE7IV2MT3TMXA, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.232042731 (593h33m34.103125766s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: expected piece hash; context canceled; EOF
Dec 30 09:02:34.232052450 (593h33m34.103135495s) [noise=true , address=45.91.116.14:4192    , piece_id=OMSJC4NYQOOV2VJZY65JUQAQ7HZANL5SEBMUT3O66OXQJJFJEG4Q, seg_pos={0 0}, upload=520344]: Long tail cancellation due to optimal threshold
Dec 30 09:02:34.232073310 (593h33m34.103156355s) [seg_pos={0 0}, upload=520344]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 09:02:34.232157620 (593h33m34.103240655s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 09:02:34.232164383 (593h33m34.103247429s) [upload=520344]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 09:02:34.521600667 (593h33m34.392683712s) [upload=520344]: Done uploading segments...
Dec 30 09:02:34.521623249 (593h33m34.392706294s) [upload=520344]: Upload finished. err: <nil>
Duration: 30.004054236s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
9 out of 60 is failed (threshold=6) [X.......................X....X.....X.........X...X.X....X.X.]
duration: 
4.088221021s
trace: 
https://jaeger.dev.storj.io/trace/41fd4dc7b7a07666
FAIL (2025-12-30 09:02:36 UTC) 
orbiter warning: context deadline exceeded
Duration: 4.088250077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [......................X.....................X...............]
duration: 
14.990250574s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:02:33 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 14.990268487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.......................X.....................X..............]
duration: 
33.549622888s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:02:18 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 33.549640611s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
9 out of 60 is failed (threshold=6) [.X.......................X....X.....X.........X...X.X....X.X]
duration: 
5.001503078s
trace: 
https://jaeger.dev.storj.io/trace/41fd4dc7b7a07666
FAIL (2025-12-30 09:02:16 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001526863s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [........................X.....................X.............]
duration: 
35.894202771s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:01:44 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 35.894231816s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [..X.......................X....X.....X.........X...X.X....X.]
duration: 
3.322287302s
trace: 
https://jaeger.dev.storj.io/trace/580b6aadb4b783f6
FAIL (2025-12-30 09:01:34 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.322327196s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
18 out of 60 is failed (threshold=3) [...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X....]
duration: 
5.982996178s
trace: 
https://jaeger.dev.storj.io/trace/3131e29c64bbf419
FAIL (2025-12-30 09:01:28 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, Piece ID: 4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA) context deadline exceeded; piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A) context deadline exceeded
======LOGS======
593h28m24.851136755s) [upload=520273]: Done starting upload
Dec 30 08:57:24.980121078 (593h28m24.851204143s) [upload=520273]: Uploading segments...
Dec 30 08:57:24.980443663 (593h28m24.851526698s) [upload=520273]: Got next segment. Inline: false
Dec 30 08:57:24.980482687 (593h28m24.851565732s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:57:24.980487837 (593h28m24.851570872s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:57:25.225107842 (593h28m25.096190877s) [seg_pos={0 0}, upload=520273]: Begin upload segment...
Dec 30 08:57:25.225159712 (593h28m25.096242747s) [seg_pos={0 0}, upload=520273]: Done begin upload segment.
Dec 30 08:57:25.225174620 (593h28m25.096257655s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225237139 (593h28m25.096320174s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225969954 (593h28m25.097052999s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226296806 (593h28m25.097379851s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226568814 (593h28m25.097651849s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226972774 (593h28m25.098055819s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227331166 (593h28m25.098414231s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227831899 (593h28m25.098914934s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228204579 (593h28m25.099287624s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228558272 (593h28m25.099641308s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229714394 (593h28m25.100797429s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229727760 (593h28m25.100810805s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231028276 (593h28m25.102111311s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231320874 (593h28m25.102403919s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.232363419 (593h28m25.103446454s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.234704546 (593h28m25.105787582s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:26.927228433 (593h28m26.798311469s) [upload=520273]: Next returned nil segment
Dec 30 08:57:26.927249664 (593h28m26.798332709s) [upload=520273]: Waiting for error group managing segments...
Dec 30 08:57:27.204176397 (593h28m27.075259432s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.214500305 (593h28m27.08558334s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.217364440 (593h28m27.088447475s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.226024359 (593h28m27.097107394s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.227263439 (593h28m27.098346484s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.233954269 (593h28m27.105037304s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.242567169 (593h28m27.113650214s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.248866344 (593h28m27.119949389s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.290843234 (593h28m27.161926289s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.302823297 (593h28m27.173906332s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.320761014 (593h28m27.191844059s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.514835152 (593h28m27.385918187s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669113833 (593h28m27.540196878s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669132809 (593h28m27.540215844s) [seg_pos={0 0}, upload=520273]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:57:27.669491422 (593h28m27.540574457s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Dec 30 08:57:27.669513925 (593h28m27.54059696s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669528764 (593h28m27.540611799s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: read tcp 188.166.225.170:47858->45.91.116.14:4192: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 30 08:57:27.669536999 (593h28m27.540620034s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669597083 (593h28m27.540680118s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection
Dec 30 08:57:27.669604678 (593h28m27.540687713s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669695170 (593h28m27.540778205s) [seg_pos={0 0}, upload=520273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:57:27.669763640 (593h28m27.540846675s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:57:27.669769792 (593h28m27.540852827s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:57:27.953145075 (593h28m27.82422811s) [upload=520273]: Done uploading segments...
Dec 30 08:57:27.953163710 (593h28m27.824246745s) [upload=520273]: Upload finished. err: <nil>
Duration: 5.983027076s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
8 out of 60 is failed (threshold=6) [...X.......................X....X.....X.........X...X.X....X]
duration: 
5.000872213s
trace: 
https://jaeger.dev.storj.io/trace/580b6aadb4b783f6
FAIL (2025-12-30 09:01:00 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000896801s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [....X.......................X....X.....X.........X...X.X....]
duration: 
3.652541858s
trace: 
https://jaeger.dev.storj.io/trace/37696adeb9d46684
FAIL (2025-12-30 09:00:42 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.65255957s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
19 out of 60 is failed (threshold=3) [X...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X...]
duration: 
5.784587737s
trace: 
https://jaeger.dev.storj.io/trace/3131e29c64bbf419
FAIL (2025-12-30 09:00:34 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, Piece ID: 4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA) context deadline exceeded; piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A) context deadline exceeded
======LOGS======
593h28m24.851136755s) [upload=520273]: Done starting upload
Dec 30 08:57:24.980121078 (593h28m24.851204143s) [upload=520273]: Uploading segments...
Dec 30 08:57:24.980443663 (593h28m24.851526698s) [upload=520273]: Got next segment. Inline: false
Dec 30 08:57:24.980482687 (593h28m24.851565732s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:57:24.980487837 (593h28m24.851570872s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:57:25.225107842 (593h28m25.096190877s) [seg_pos={0 0}, upload=520273]: Begin upload segment...
Dec 30 08:57:25.225159712 (593h28m25.096242747s) [seg_pos={0 0}, upload=520273]: Done begin upload segment.
Dec 30 08:57:25.225174620 (593h28m25.096257655s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225237139 (593h28m25.096320174s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225969954 (593h28m25.097052999s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226296806 (593h28m25.097379851s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226568814 (593h28m25.097651849s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226972774 (593h28m25.098055819s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227331166 (593h28m25.098414231s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227831899 (593h28m25.098914934s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228204579 (593h28m25.099287624s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228558272 (593h28m25.099641308s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229714394 (593h28m25.100797429s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229727760 (593h28m25.100810805s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231028276 (593h28m25.102111311s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231320874 (593h28m25.102403919s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.232363419 (593h28m25.103446454s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.234704546 (593h28m25.105787582s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:26.927228433 (593h28m26.798311469s) [upload=520273]: Next returned nil segment
Dec 30 08:57:26.927249664 (593h28m26.798332709s) [upload=520273]: Waiting for error group managing segments...
Dec 30 08:57:27.204176397 (593h28m27.075259432s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.214500305 (593h28m27.08558334s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.217364440 (593h28m27.088447475s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.226024359 (593h28m27.097107394s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.227263439 (593h28m27.098346484s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.233954269 (593h28m27.105037304s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.242567169 (593h28m27.113650214s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.248866344 (593h28m27.119949389s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.290843234 (593h28m27.161926289s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.302823297 (593h28m27.173906332s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.320761014 (593h28m27.191844059s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.514835152 (593h28m27.385918187s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669113833 (593h28m27.540196878s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669132809 (593h28m27.540215844s) [seg_pos={0 0}, upload=520273]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:57:27.669491422 (593h28m27.540574457s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Dec 30 08:57:27.669513925 (593h28m27.54059696s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669528764 (593h28m27.540611799s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: read tcp 188.166.225.170:47858->45.91.116.14:4192: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 30 08:57:27.669536999 (593h28m27.540620034s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669597083 (593h28m27.540680118s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection
Dec 30 08:57:27.669604678 (593h28m27.540687713s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669695170 (593h28m27.540778205s) [seg_pos={0 0}, upload=520273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:57:27.669763640 (593h28m27.540846675s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:57:27.669769792 (593h28m27.540852827s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:57:27.953145075 (593h28m27.82422811s) [upload=520273]: Done uploading segments...
Dec 30 08:57:27.953163710 (593h28m27.824246745s) [upload=520273]: Upload finished. err: <nil>
Duration: 5.784613786s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.........................X.....................X............]
duration: 
26.241140939s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 09:00:25 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 26.241167602s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [.....X.......................X....X.....X.........X...X.X...]
duration: 
3.606172144s
trace: 
https://jaeger.dev.storj.io/trace/37696adeb9d46684
FAIL (2025-12-30 09:00:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.606199796s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..........................X.....................X...........]
duration: 
15.228466423s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:59:48 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 15.228492943s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
20 out of 60 is failed (threshold=3) [XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X..]
duration: 
5.923247977s
trace: 
https://jaeger.dev.storj.io/trace/3131e29c64bbf419
FAIL (2025-12-30 08:59:31 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, Piece ID: 4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA) context deadline exceeded; piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A) context deadline exceeded
======LOGS======
593h28m24.851136755s) [upload=520273]: Done starting upload
Dec 30 08:57:24.980121078 (593h28m24.851204143s) [upload=520273]: Uploading segments...
Dec 30 08:57:24.980443663 (593h28m24.851526698s) [upload=520273]: Got next segment. Inline: false
Dec 30 08:57:24.980482687 (593h28m24.851565732s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:57:24.980487837 (593h28m24.851570872s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:57:25.225107842 (593h28m25.096190877s) [seg_pos={0 0}, upload=520273]: Begin upload segment...
Dec 30 08:57:25.225159712 (593h28m25.096242747s) [seg_pos={0 0}, upload=520273]: Done begin upload segment.
Dec 30 08:57:25.225174620 (593h28m25.096257655s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225237139 (593h28m25.096320174s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225969954 (593h28m25.097052999s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226296806 (593h28m25.097379851s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226568814 (593h28m25.097651849s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226972774 (593h28m25.098055819s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227331166 (593h28m25.098414231s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227831899 (593h28m25.098914934s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228204579 (593h28m25.099287624s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228558272 (593h28m25.099641308s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229714394 (593h28m25.100797429s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229727760 (593h28m25.100810805s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231028276 (593h28m25.102111311s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231320874 (593h28m25.102403919s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.232363419 (593h28m25.103446454s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.234704546 (593h28m25.105787582s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:26.927228433 (593h28m26.798311469s) [upload=520273]: Next returned nil segment
Dec 30 08:57:26.927249664 (593h28m26.798332709s) [upload=520273]: Waiting for error group managing segments...
Dec 30 08:57:27.204176397 (593h28m27.075259432s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.214500305 (593h28m27.08558334s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.217364440 (593h28m27.088447475s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.226024359 (593h28m27.097107394s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.227263439 (593h28m27.098346484s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.233954269 (593h28m27.105037304s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.242567169 (593h28m27.113650214s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.248866344 (593h28m27.119949389s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.290843234 (593h28m27.161926289s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.302823297 (593h28m27.173906332s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.320761014 (593h28m27.191844059s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.514835152 (593h28m27.385918187s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669113833 (593h28m27.540196878s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669132809 (593h28m27.540215844s) [seg_pos={0 0}, upload=520273]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:57:27.669491422 (593h28m27.540574457s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Dec 30 08:57:27.669513925 (593h28m27.54059696s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669528764 (593h28m27.540611799s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: read tcp 188.166.225.170:47858->45.91.116.14:4192: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 30 08:57:27.669536999 (593h28m27.540620034s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669597083 (593h28m27.540680118s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection
Dec 30 08:57:27.669604678 (593h28m27.540687713s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669695170 (593h28m27.540778205s) [seg_pos={0 0}, upload=520273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:57:27.669763640 (593h28m27.540846675s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:57:27.669769792 (593h28m27.540852827s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:57:27.953145075 (593h28m27.82422811s) [upload=520273]: Done uploading segments...
Dec 30 08:57:27.953163710 (593h28m27.824246745s) [upload=520273]: Upload finished. err: <nil>
Duration: 5.923277133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...........................X.....................X..........]
duration: 
30.301401318s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:59:16 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 30.301515345s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [......X.......................X....X.....X.........X...X.X..]
duration: 
3.721803204s
trace: 
https://jaeger.dev.storj.io/trace/37696adeb9d46684
FAIL (2025-12-30 08:59:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.721831326s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [.......X.......................X....X.....X.........X...X.X.]
duration: 
3.542723876s
trace: 
https://jaeger.dev.storj.io/trace/37696adeb9d46684
FAIL (2025-12-30 08:58:24 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.542760939s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
20 out of 60 is failed (threshold=3) [.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X.]
duration: 
6.0575511s
trace: 
https://jaeger.dev.storj.io/trace/3131e29c64bbf419
FAIL (2025-12-30 08:58:22 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, Piece ID: 4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA) context deadline exceeded; piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A) context deadline exceeded
======LOGS======
593h28m24.851136755s) [upload=520273]: Done starting upload
Dec 30 08:57:24.980121078 (593h28m24.851204143s) [upload=520273]: Uploading segments...
Dec 30 08:57:24.980443663 (593h28m24.851526698s) [upload=520273]: Got next segment. Inline: false
Dec 30 08:57:24.980482687 (593h28m24.851565732s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:57:24.980487837 (593h28m24.851570872s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:57:25.225107842 (593h28m25.096190877s) [seg_pos={0 0}, upload=520273]: Begin upload segment...
Dec 30 08:57:25.225159712 (593h28m25.096242747s) [seg_pos={0 0}, upload=520273]: Done begin upload segment.
Dec 30 08:57:25.225174620 (593h28m25.096257655s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225237139 (593h28m25.096320174s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225969954 (593h28m25.097052999s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226296806 (593h28m25.097379851s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226568814 (593h28m25.097651849s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226972774 (593h28m25.098055819s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227331166 (593h28m25.098414231s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227831899 (593h28m25.098914934s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228204579 (593h28m25.099287624s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228558272 (593h28m25.099641308s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229714394 (593h28m25.100797429s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229727760 (593h28m25.100810805s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231028276 (593h28m25.102111311s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231320874 (593h28m25.102403919s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.232363419 (593h28m25.103446454s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.234704546 (593h28m25.105787582s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:26.927228433 (593h28m26.798311469s) [upload=520273]: Next returned nil segment
Dec 30 08:57:26.927249664 (593h28m26.798332709s) [upload=520273]: Waiting for error group managing segments...
Dec 30 08:57:27.204176397 (593h28m27.075259432s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.214500305 (593h28m27.08558334s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.217364440 (593h28m27.088447475s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.226024359 (593h28m27.097107394s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.227263439 (593h28m27.098346484s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.233954269 (593h28m27.105037304s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.242567169 (593h28m27.113650214s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.248866344 (593h28m27.119949389s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.290843234 (593h28m27.161926289s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.302823297 (593h28m27.173906332s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.320761014 (593h28m27.191844059s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.514835152 (593h28m27.385918187s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669113833 (593h28m27.540196878s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669132809 (593h28m27.540215844s) [seg_pos={0 0}, upload=520273]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:57:27.669491422 (593h28m27.540574457s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Dec 30 08:57:27.669513925 (593h28m27.54059696s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669528764 (593h28m27.540611799s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: read tcp 188.166.225.170:47858->45.91.116.14:4192: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 30 08:57:27.669536999 (593h28m27.540620034s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669597083 (593h28m27.540680118s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection
Dec 30 08:57:27.669604678 (593h28m27.540687713s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669695170 (593h28m27.540778205s) [seg_pos={0 0}, upload=520273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:57:27.669763640 (593h28m27.540846675s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:57:27.669769792 (593h28m27.540852827s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:57:27.953145075 (593h28m27.82422811s) [upload=520273]: Done uploading segments...
Dec 30 08:57:27.953163710 (593h28m27.824246745s) [upload=520273]: Upload finished. err: <nil>
Duration: 6.057583303s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
7 out of 60 is failed (threshold=6) [........X.......................X....X.....X.........X...X.X]
duration: 
5.001388811s
trace: 
https://jaeger.dev.storj.io/trace/37696adeb9d46684
FAIL (2025-12-30 08:58:03 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001465226s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [............................X.....................X.........]
duration: 
26.367295347s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:57:59 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 26.367312259s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
21 out of 60 is failed (threshold=3) [X.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......X]
duration: 
30.00175742s
trace: 
https://jaeger.dev.storj.io/trace/3131e29c64bbf419
FAIL (2025-12-30 08:57:54 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, Piece ID: 4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA) context deadline exceeded; piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A) context deadline exceeded
======LOGS======
593h28m24.851136755s) [upload=520273]: Done starting upload
Dec 30 08:57:24.980121078 (593h28m24.851204143s) [upload=520273]: Uploading segments...
Dec 30 08:57:24.980443663 (593h28m24.851526698s) [upload=520273]: Got next segment. Inline: false
Dec 30 08:57:24.980482687 (593h28m24.851565732s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:57:24.980487837 (593h28m24.851570872s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:57:25.225107842 (593h28m25.096190877s) [seg_pos={0 0}, upload=520273]: Begin upload segment...
Dec 30 08:57:25.225159712 (593h28m25.096242747s) [seg_pos={0 0}, upload=520273]: Done begin upload segment.
Dec 30 08:57:25.225174620 (593h28m25.096257655s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225237139 (593h28m25.096320174s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.225969954 (593h28m25.097052999s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226296806 (593h28m25.097379851s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226568814 (593h28m25.097651849s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.226972774 (593h28m25.098055819s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227331166 (593h28m25.098414231s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.227831899 (593h28m25.098914934s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228204579 (593h28m25.099287624s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.228558272 (593h28m25.099641308s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229714394 (593h28m25.100797429s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.229727760 (593h28m25.100810805s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231028276 (593h28m25.102111311s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.231320874 (593h28m25.102403919s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.232363419 (593h28m25.103446454s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:25.234704546 (593h28m25.105787582s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Uploading piece...
Dec 30 08:57:26.927228433 (593h28m26.798311469s) [upload=520273]: Next returned nil segment
Dec 30 08:57:26.927249664 (593h28m26.798332709s) [upload=520273]: Waiting for error group managing segments...
Dec 30 08:57:27.204176397 (593h28m27.075259432s) [noise=true , address=145.79.227.36:4518   , piece_id=W72ZUCIUEZIRG4HPCGE4GBFQHFNZDRU5C2RBWGL5BSL6I42GLMTA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.214500305 (593h28m27.08558334s) [noise=true , address=145.79.227.50:4375   , piece_id=L5LKJ2IKBPGOJZSYDR3PQEUEJER2K6577DOKHGK2TYLWCUELQCAQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.217364440 (593h28m27.088447475s) [noise=true , address=145.79.227.15:4876   , piece_id=JC7HSWEFTPIXRSP5SDTGSA7GMIQFNXMICUMJZHYMWTELITKULCRQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.226024359 (593h28m27.097107394s) [noise=true , address=145.79.227.4:4185    , piece_id=CG6CH65COS5LI6J3Y4I3NYZBIBSWMZXTO7KO7G6Z4A7N65YKDKLQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.227263439 (593h28m27.098346484s) [noise=true , address=145.79.227.9:4365    , piece_id=RKSHKUWNFZAZLUNJQH7YHFOCPLVFER44L3LVV6J7YDQ35KQCMFLA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.233954269 (593h28m27.105037304s) [noise=true , address=145.79.227.14:4837   , piece_id=ZYOP5COCKCY5HTWYPKTHCN3EOMMCLXWIXWOOCER7V7JYLBBWPZJA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.242567169 (593h28m27.113650214s) [noise=true , address=145.79.227.45:4230   , piece_id=PX2MILMDMN3IJF6HRXBGZ6YIDSZ3UNGXXYSOJ7J6KJOFHEWLMQHA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.248866344 (593h28m27.119949389s) [noise=true , address=145.79.227.47:4266   , piece_id=4JGO7ZKW7ZOMTNGGQUKFYWDXBL4XM4PCK2YKYACDGPREUG6D6FSA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.290843234 (593h28m27.161926289s) [noise=true , address=145.79.227.16:4909   , piece_id=KLTHE6SAZO3YQLPLQUL5QUOOF2SU4VXOVP57B56SZ7HR346IOJRA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.302823297 (593h28m27.173906332s) [noise=true , address=145.79.227.7:4295    , piece_id=YEZ5IUL3LIOPPUIS5APYR4QDKTE753OCDIQS3MXOL674UAC7R2AA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.320761014 (593h28m27.191844059s) [noise=true , address=145.79.227.27:4691   , piece_id=N4WOWTR5WRZ6NYRVBOFUBQBK7TY47K6AY623BO7K6FQCYYUM6AQQ, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.514835152 (593h28m27.385918187s) [noise=true , address=77.90.31.10:4517     , piece_id=VARAV7FGWUT2X7YG7342LDGWDEWAWMA7527LOCWZFJ6PXFMME76A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669113833 (593h28m27.540196878s) [noise=true , address=45.91.116.11:4117    , piece_id=4XFSKPP43UYFKTXBMUYRNABIUZEMGPMFN26UUEFUWAWLSNEA5YIA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: <nil>
Dec 30 08:57:27.669132809 (593h28m27.540215844s) [seg_pos={0 0}, upload=520273]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:57:27.669491422 (593h28m27.540574457s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Dec 30 08:57:27.669513925 (593h28m27.54059696s) [noise=true , address=45.91.116.5:4297     , piece_id=JDEY25PQM3VKDOWPCVCITK4WT22NSRQGBV72JWVB7VZRKDV5HGAA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669528764 (593h28m27.540611799s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs): protocol: read tcp 188.166.225.170:47858->45.91.116.14:4192: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 30 08:57:27.669536999 (593h28m27.540620034s) [noise=true , address=45.91.116.14:4192    , piece_id=MP6OYIYDTBZVPDOYHJQTO3KDB2T2L4XKG6DLRODA22CM2KUR2B3A, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669597083 (593h28m27.540680118s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40916->77.90.31.21:4880: use of closed network connection
Dec 30 08:57:27.669604678 (593h28m27.540687713s) [noise=true , address=77.90.31.21:4880     , piece_id=GX7I33GF3FINHIRXOJU3LOGZX3CEZKYSPZ4OSMFAOTQ2D4JIP3QA, seg_pos={0 0}, upload=520273]: Long tail cancellation due to optimal threshold
Dec 30 08:57:27.669695170 (593h28m27.540778205s) [seg_pos={0 0}, upload=520273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:57:27.669763640 (593h28m27.540846675s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:57:27.669769792 (593h28m27.540852827s) [upload=520273]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:57:27.953145075 (593h28m27.82422811s) [upload=520273]: Done uploading segments...
Dec 30 08:57:27.953163710 (593h28m27.824246745s) [upload=520273]: Upload finished. err: <nil>
Duration: 30.001806493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.............................X.....................X........]
duration: 
36.86510967s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:57:32 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 36.865136351s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
6 out of 60 is failed (threshold=6) [.........X.......................X....X.....X.........X...X.]
duration: 
3.5285405s
trace: 
https://jaeger.dev.storj.io/trace/56d7eed349344de3
FAIL (2025-12-30 08:57:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.528606905s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
6 out of 60 is failed (threshold=6) [..........X.......................X....X.....X.........X...X]
duration: 
5.001226911s
trace: 
https://jaeger.dev.storj.io/trace/56d7eed349344de3
FAIL (2025-12-30 08:56:30 UTC) 
orbiter warning: context deadline exceeded
Duration: 5.001249202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [..............................X.....................X.......]
duration: 
28.589443389s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:56:29 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 28.589474088s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
21 out of 60 is failed (threshold=3) [XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X......]
duration: 
5.873157517s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:56:23 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 5.873175582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
21 out of 60 is failed (threshold=3) [.XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X.....]
duration: 
6.0432718s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:55:45 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 6.043300425s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...............................X.....................X......]
duration: 
30.429905617s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:55:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 30.42991865s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
5 out of 60 is failed (threshold=6) [...........X.......................X....X.....X.........X...]
duration: 
3.52679689s
trace: 
https://jaeger.dev.storj.io/trace/7f438d7123c7ef08
FAIL (2025-12-30 08:55:27 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.526919804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [X...............................X.....................X.....]
duration: 
39.270088728s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:55:07 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 39.270113366s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
5 out of 60 is failed (threshold=6) [............X.......................X....X.....X.........X..]
duration: 
3.555870671s
trace: 
https://jaeger.dev.storj.io/trace/7f438d7123c7ef08
FAIL (2025-12-30 08:55:07 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.555887352s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
21 out of 60 is failed (threshold=3) [..XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X....]
duration: 
5.90121762s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:54:38 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 5.90125508s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
5 out of 60 is failed (threshold=6) [.............X.......................X....X.....X.........X.]
duration: 
3.441470595s
trace: 
https://jaeger.dev.storj.io/trace/7f438d7123c7ef08
FAIL (2025-12-30 08:54:36 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.44149909s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
21 out of 60 is failed (threshold=3) [...XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X...]
duration: 
6.74354304s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:54:07 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 6.743577787s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [.X...............................X.....................X....]
duration: 
32.252271744s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:54:07 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 32.252297683s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
22 out of 60 is failed (threshold=3) [X...XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X..]
duration: 
23.583166675s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:53:40 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 23.583195439s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
5 out of 60 is failed (threshold=6) [..............X.......................X....X.....X.........X]
duration: 
5.00131104s
trace: 
https://jaeger.dev.storj.io/trace/7f438d7123c7ef08
FAIL (2025-12-30 08:53:29 UTC) 
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001332681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [..X...............................X.....................X...]
duration: 
42.502228121s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:53:28 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 42.50225369s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
4 out of 60 is failed (threshold=6) [...............X.......................X....X.....X.........]
duration: 
3.526870641s
trace: 
https://jaeger.dev.storj.io/trace/42934eb5b36f118d
FAIL (2025-12-30 08:53:05 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.526936786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
22 out of 60 is failed (threshold=3) [.X...XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X.]
duration: 
6.158069217s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:53:01 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 6.158088544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
23 out of 60 is failed (threshold=3) [X.X...XX.XX...X...X...X.X.X..X.XXX...X.X.X.X.........X.X.X.X]
duration: 
30.002736975s
trace: 
https://jaeger.dev.storj.io/trace/3a08d86b60635921
FAIL (2025-12-30 08:52:54 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, Piece ID: KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA) context deadline exceeded
======LOGS======
 08:52:25.480758080 (593h23m25.351841135s) [upload=520199]: Starting upload
Dec 30 08:52:25.480834846 (593h23m25.351917921s) [upload=520199]: Done starting upload
Dec 30 08:52:25.481066347 (593h23m25.352149382s) [upload=520199]: Uploading segments...
Dec 30 08:52:25.481110801 (593h23m25.352193836s) [upload=520199]: Got next segment. Inline: false
Dec 30 08:52:25.481137933 (593h23m25.352220978s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 30 08:52:25.481142532 (593h23m25.352225567s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 30 08:52:25.726063941 (593h23m25.597146986s) [seg_pos={0 0}, upload=520199]: Begin upload segment...
Dec 30 08:52:25.726136809 (593h23m25.597219854s) [seg_pos={0 0}, upload=520199]: Done begin upload segment.
Dec 30 08:52:25.726185702 (593h23m25.597268747s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.726725420 (593h23m25.597808465s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.727538758 (593h23m25.598621794s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728078677 (593h23m25.599161712s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728183016 (593h23m25.599266061s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728433673 (593h23m25.599516718s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728630368 (593h23m25.599713413s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728797926 (593h23m25.599880961s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.728965094 (593h23m25.60004814s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729179553 (593h23m25.600262598s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729594763 (593h23m25.600677808s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.729607338 (593h23m25.600690373s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730256544 (593h23m25.601339589s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.730724265 (593h23m25.6018073s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.731124847 (593h23m25.602207882s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:25.732280307 (593h23m25.603363342s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Uploading piece...
Dec 30 08:52:27.337973407 (593h23m27.209056452s) [upload=520199]: Next returned nil segment
Dec 30 08:52:27.337991041 (593h23m27.209074076s) [upload=520199]: Waiting for error group managing segments...
Dec 30 08:52:27.719226103 (593h23m27.590309148s) [noise=true , address=145.79.227.9:4365    , piece_id=OG2MWYVOLC3VQRIDQQV7RGXRAZ4C4IHCJIMQE4V6KGMAEAOPN5JQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.722058597 (593h23m27.593141642s) [noise=true , address=145.79.227.38:4977   , piece_id=GMJKODWLUZF5UJCGL5KTHRDABBI3SMP3NWZAI3POMHZNTP3FDFXA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.733106553 (593h23m27.604189588s) [noise=true , address=145.79.227.48:4303   , piece_id=ID4X4ZPLMBWWOAZP4P4SVGNJZ5FNDBG66FJ7KTLV3UOGN53M7M3A, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.735225438 (593h23m27.606308483s) [noise=true , address=145.79.227.12:4767   , piece_id=O4NO6QWUPGO3XZZXN2565VQOAVAATOGNCBULYU4ZDMGLOXYYAGPQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.737523574 (593h23m27.608606619s) [noise=true , address=145.79.227.7:4295    , piece_id=OL7EV3KS2VNAIJ2X2KBFKKGM5GYZGU562RSU52JMXOVEVQZGEUDQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739302061 (593h23m27.610385106s) [noise=true , address=145.79.227.50:4374   , piece_id=WPM6F4INUYL6VBX23QILI2KYDTBEGB7F3BED7ZROTMOW6Y6NQZCA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.739408905 (593h23m27.61049194s) [noise=true , address=145.79.227.51:4410   , piece_id=ABMR4QIJ4NKJRPOABLPGHLHDCD3QOYH5HCNRKBCTAEYEAZKB5SRQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.755068693 (593h23m27.626151738s) [noise=true , address=145.79.227.11:4731   , piece_id=RCJ4APOIBKUGYD7N6NA565SSPDYTQVYO22HAZP4ZPRGGST4QFD6Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.763860915 (593h23m27.634943951s) [noise=true , address=145.79.227.20:4438   , piece_id=H7AVKI67IDY7YH67XQ55IZMUQ3ODT2UJTI6ET66GTWRWINW75QYA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.787301309 (593h23m27.658384344s) [noise=true , address=145.79.227.13:4803   , piece_id=HWIZEEXHMCTVRCLTZ6AVO6MM6SQYLPBHUUZ2ARDCACG6MPXHXOMA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:27.804569451 (593h23m27.675652496s) [noise=true , address=145.79.227.37:4565   , piece_id=VEFUATOIDQJFKR72IMHAS267TWDVB776KZNGE5NXZ2KNKOR7Z64Q, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.035108183 (593h23m27.906191218s) [noise=true , address=45.91.116.13:4156    , piece_id=KZMEOCO7CWLDO72WFE66Y2O6ZAZB3S55G556YPOXPGCDHVIMT4TA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067811396 (593h23m27.938894441s) [noise=true , address=45.91.116.12:4228    , piece_id=TIQRR6XH3AD4OLSWC57VTUIJWC43LUBORMXLNBZMMLVA5GNVX6MA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: <nil>
Dec 30 08:52:28.067831734 (593h23m27.938914769s) [seg_pos={0 0}, upload=520199]: Segment reached optimal threshold of 13 pieces.
Dec 30 08:52:28.068010505 (593h23m27.93909355s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34386->45.91.116.11:4118: use of closed network connection
Dec 30 08:52:28.068015955 (593h23m27.93909899s) [noise=true , address=45.91.116.11:4118    , piece_id=5FZM6C67GWACD4EXCQRABGPGYMGUDOTI5MBU4Q2EYQJYDYN3KQCQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068218511 (593h23m27.939301576s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068250281 (593h23m27.939333326s) [noise=true , address=77.90.31.21:4878     , piece_id=2IMENAHZB4ACCHVYVSXHBM7H45OTEACGO5RSBPEY37Q4KCZ6T7EA, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068269488 (593h23m27.939352523s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Dec 30 08:52:28.068276652 (593h23m27.939359687s) [noise=true , address=45.91.116.4:4937     , piece_id=T2LCVRXHLD4MZPYMA36ZXN4A7FA3CIKJ6LXFL3UGWEPEWT5UCZVQ, seg_pos={0 0}, upload=520199]: Long tail cancellation due to optimal threshold
Dec 30 08:52:28.068299736 (593h23m27.939382781s) [seg_pos={0 0}, upload=520199]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 30 08:52:28.068384166 (593h23m27.939467201s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:52:28.068389577 (593h23m27.939472612s) [upload=520199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:52:28.354664991 (593h23m28.225748036s) [upload=520199]: Done uploading segments...
Dec 30 08:52:28.354690600 (593h23m28.225773645s) [upload=520199]: Upload finished. err: <nil>
Duration: 30.002752745s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [...X...............................X.....................X..]
duration: 
15.213901968s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:52:24 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 15.213919441s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
4 out of 60 is failed (threshold=6) [................X.......................X....X.....X........]
duration: 
3.46166008s
trace: 
https://jaeger.dev.storj.io/trace/42934eb5b36f118d
FAIL (2025-12-30 08:52:09 UTC) 
orbiter warning: context deadline exceeded
Duration: 3.461691118s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [....X...............................X.....................X.]
duration: 
34.194210205s
trace: 
https://jaeger.dev.storj.io/trace/e14e3f77cfefb30
FAIL (2025-12-30 08:52:08 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: quiescence
======LOGS======
227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807118843 (593h21m35.678201878s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807337019 (593h21m35.678420054s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807436869 (593h21m35.678519904s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.807815018 (593h21m35.678898053s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808170957 (593h21m35.679253992s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808550199 (593h21m35.679633234s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808897170 (593h21m35.679980205s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.808938168 (593h21m35.680021213s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.809241615 (593h21m35.68032466s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.810184420 (593h21m35.681267455s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.813924442 (593h21m35.685007477s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.815805704 (593h21m35.68688878s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.817128213 (593h21m35.688211248s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:35.818412769 (593h21m35.689495814s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Uploading piece...
Dec 30 08:50:37.246488707 (593h21m37.117571742s) [upload=520173]: Next returned nil segment
Dec 30 08:50:37.246504797 (593h21m37.117587842s) [upload=520173]: Waiting for error group managing segments...
Dec 30 08:50:37.617119537 (593h21m37.488202582s) [noise=true , address=64.49.13.27:4964     , piece_id=3IOD6GOR2JUQHZIWAQAEXBBGKJCBITIZE6MRANNS4TFGMIGN4LNA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.656276388 (593h21m37.527359423s) [noise=true , address=64.49.13.26:4956     , piece_id=SU2QROGRLIS4YQFE2RES3PH2S32BHVI3TXF7U2JGDYC3K366KNNQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.692596076 (593h21m37.563679111s) [noise=true , address=145.79.227.17:4987   , piece_id=26ZUGT6NIH4IOR6SKC6VWFZMRZNVOELETN7GGJG632QJWLKVRYYQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.733155218 (593h21m37.604238253s) [noise=true , address=145.79.227.29:4955   , piece_id=JCX2CSRZEP2PMGJPLGWBHV4UORVX3RE3ZOV656PY5RU4HF6ALH2A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.905397355 (593h21m37.77648039s) [noise=true , address=77.90.31.14:4683     , piece_id=2GLICJXR46KXKR2NWUQICK4FL42HHF6UAR4TXVFXR3NB32YQDUHA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.951372488 (593h21m37.822455523s) [noise=true , address=77.90.31.12:4594     , piece_id=NWJGVMQ7CHOREBNQ33IFVKBQUXDPU7RDS2RS4GJISEFYJCFTJA7A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:37.963800493 (593h21m37.834883529s) [noise=true , address=77.90.31.15:4713     , piece_id=SBSHVYFUZHPAVVRPF7CIH73RT7A3EDX7UTZM6CRSDM5XUBURA3AQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.149230773 (593h21m38.020313808s) [noise=true , address=145.79.227.40:4055   , piece_id=A7DPT5GGHF3ZRS3A4SWTBIQOGXP5NHFE23BHNGWYQSTOEJXD33QQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.180461882 (593h21m38.051544917s) [noise=true , address=45.91.116.13:4186    , piece_id=E5PRUUMJWUCSE33VQZ5M62GIOIRW4WMLZLRVIPWWEUVSNL6WAXUA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.195839091 (593h21m38.066922136s) [noise=true , address=77.90.31.21:4902     , piece_id=5DSA5INCNXIXC7V4RJX7DTRPWN7IV2IIXHS3UIERFMB2Z6QBUR3A, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.557140909 (593h21m38.428223954s) [noise=true , address=145.79.227.44:4202   , piece_id=SJGLBUIHJOVVRCJHYHIZ333TUZ7HCZWIJEWRMUP235YGCMHBFAZA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.564815133 (593h21m38.435898178s) [noise=true , address=193.149.164.143:4696 , piece_id=U26EFERLZTQ4635BUUF4ZHQ5AK4LV4EPYBYFK75HSGAPRJSQV3HA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680119209 (593h21m38.551202255s) [noise=true , address=145.79.227.30:4765   , piece_id=YHPLTJX7A7TEZ6YUDERQLEAIUAZXILETKG6RQI2WA2CPJIA3BQCQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680487441 (593h21m38.551570486s) [noise=true , address=145.79.227.34:4934   , piece_id=NRB3NAOI2GZKRBUU7EVVH3BD3W7YKTUY2JCNGUI6SCWGE6A2D5CQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: <nil>
Dec 30 08:50:38.680509433 (593h21m38.551592468s) [seg_pos={0 0}, upload=520173]: Segment reached optimal threshold of 14 pieces.
Dec 30 08:50:38.680771542 (593h21m38.551854577s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48684->64.49.13.169:4707: use of closed network connection
Dec 30 08:50:38.680782382 (593h21m38.551865427s) [noise=true , address=64.49.13.169:4707    , piece_id=KMLWVT7YRKYSGOL76PSZQGXWCRDKOGFIN7CSKBWNM7KDVYN7XQPA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.680806418 (593h21m38.551889453s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF
Dec 30 08:50:38.680814443 (593h21m38.551897488s) [noise=true , address=145.79.227.35:4496   , piece_id=4SUIGSUGSABL76I7ZMFXBFMFSJX3EUREZUPLNEBEHCMZIR2MZIJA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681009154 (593h21m38.5520922s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX): protocol: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57998->145.79.227.43:4170: use of closed network connection
Dec 30 08:50:38.681014925 (593h21m38.55209797s) [noise=true , address=145.79.227.43:4170   , piece_id=OTJQZRRMCTAGA6TSNK3XRXVPFJZTD3NSDRPSSLH7F2EE6YTWCNBQ, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681264851 (593h21m38.552347886s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp): protocol: context canceled; context canceled
Dec 30 08:50:38.681273046 (593h21m38.552356081s) [noise=true , address=145.79.227.33:4881   , piece_id=Y533AVONHIRVWMP3IIK4W3T3OPYKOEVDAM5EF742HDJOEFJPB3YA, seg_pos={0 0}, upload=520173]: Long tail cancellation due to optimal threshold
Dec 30 08:50:38.681288476 (593h21m38.552371511s) [seg_pos={0 0}, upload=520173]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 30 08:50:38.681367637 (593h21m38.552450672s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 30 08:50:38.681372125 (593h21m38.55245517s) [upload=520173]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 30 08:50:38.968146240 (593h21m38.839229275s) [upload=520173]: Done uploading segments...
Dec 30 08:50:38.968170716 (593h21m38.839253762s) [upload=520173]: Upload finished. err: <nil>
Duration: 34.19422865s core uplink us-select-1 us1-placement-20