Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X........................X........................XX.......] duration: 22.616327426s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:29:18 UTC) orbiter warning: context deadline exceeded Duration: 22.616352373s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXXXX..X] duration: 5.001607985s trace: https://jaeger.dev.storj.io/trace/5b4cebf03f3a17d0 FAIL (2026-01-15 10:28:49 UTC) orbiter warning: context deadline exceeded Duration: 5.001633142s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.712647964s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:28:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.712679384s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X........................X........................XX......] duration: 18.018723436s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:28:28 UTC) orbiter warning: context deadline exceeded Duration: 18.018749345s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXXXX..] duration: 3.864491375s trace: https://jaeger.dev.storj.io/trace/3861ca93e62a8d79 FAIL (2026-01-15 10:27:50 UTC) orbiter warning: context deadline exceeded Duration: 3.864514628s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X........................X........................XX.....] duration: 17.224487777s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:27:37 UTC) orbiter warning: context deadline exceeded Duration: 17.224510511s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.777162862s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:27:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.77718805s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 55 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXXXX.] duration: 3.689498345s trace: https://jaeger.dev.storj.io/trace/3861ca93e62a8d79 FAIL (2026-01-15 10:27:06 UTC) orbiter warning: context deadline exceeded Duration: 3.689522001s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.823435849s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:26:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.823458985s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXXXX] duration: 5.001399789s trace: https://jaeger.dev.storj.io/trace/3861ca93e62a8d79 FAIL (2026-01-15 10:26:26 UTC) orbiter warning: context deadline exceeded Duration: 5.001424025s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X........................X........................XX....] duration: 13.496321302s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:26:21 UTC) orbiter warning: context deadline exceeded Duration: 13.496343475s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.835696964s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:26:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.83572059s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X........................X........................XX...] duration: 15.942684638s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:25:47 UTC) orbiter warning: context deadline exceeded Duration: 15.942713103s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.771912245s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:25:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.771945498s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXXX] duration: 5.001339885s trace: https://jaeger.dev.storj.io/trace/557c856eacab900d FAIL (2026-01-15 10:25:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001354022s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.874260534s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:25:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.874290803s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X........................X........................XX..] duration: 14.749715716s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:25:06 UTC) orbiter warning: context deadline exceeded Duration: 14.749742027s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.784923389s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:24:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.784942176s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXXX] duration: 5.000822912s trace: https://jaeger.dev.storj.io/trace/501a12decbe4a039 FAIL (2026-01-15 10:24:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00084829s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X........................X........................XX.] duration: 17.505008519s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:24:17 UTC) orbiter warning: context deadline exceeded Duration: 17.505043966s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X........................X........................XX] duration: 30.0012583s trace: https://jaeger.dev.storj.io/trace/533ee539a27d5e6f FAIL (2026-01-15 10:23:59 UTC) orbiter warning: context deadline exceeded Duration: 30.001281683s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.755468366s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:23:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.755492052s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X........................X........................X] duration: 30.001535024s trace: https://jaeger.dev.storj.io/trace/6116492af76813bf FAIL (2026-01-15 10:23:27 UTC) orbiter warning: context deadline exceeded Duration: 30.001556405s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXXX] duration: 5.000823069s trace: https://jaeger.dev.storj.io/trace/34e9399d831293ae FAIL (2026-01-15 10:23:19 UTC) orbiter warning: context deadline exceeded Duration: 5.000833518s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.784148656s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:23:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.784173484s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXXX] duration: 5.001025257s trace: https://jaeger.dev.storj.io/trace/725af28a64ef8ab5 FAIL (2026-01-15 10:22:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001047067s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X........................X........................] duration: 17.551175108s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:22:38 UTC) orbiter warning: context deadline exceeded Duration: 17.551215023s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.746463147s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:22:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.746484377s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X........................X.......................] duration: 16.255457976s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:21:59 UTC) orbiter warning: context deadline exceeded Duration: 16.255506078s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXXX] duration: 5.000618381s trace: https://jaeger.dev.storj.io/trace/2df315453c1aea3 FAIL (2026-01-15 10:21:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000639241s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X........................X......................] duration: 21.0801228s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:21:42 UTC) orbiter warning: context deadline exceeded Duration: 21.080145021s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 30.433264733s trace: https://jaeger.dev.storj.io/trace/6695e64327811e17 FAIL (2026-01-15 10:21:37 UTC) orbiter warning: context deadline exceeded Duration: 30.433290722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.677621079s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:21:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.677649214s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X........................X.....................] duration: 14.177908449s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:21:07 UTC) orbiter warning: context deadline exceeded Duration: 14.177951011s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXXX] duration: 5.00196779s trace: https://jaeger.dev.storj.io/trace/4cc1e5739d89efd1 FAIL (2026-01-15 10:20:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002022747s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 13.187353021s trace: https://jaeger.dev.storj.io/trace/6695e64327811e17 FAIL (2026-01-15 10:20:53 UTC) orbiter warning: context deadline exceeded Duration: 13.187431811s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXXX] duration: 5.00109728s trace: https://jaeger.dev.storj.io/trace/59cdca5d1466f881 FAIL (2026-01-15 10:20:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001118211s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.............X........................X....................] duration: 14.478380352s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:20:13 UTC) orbiter warning: context deadline exceeded Duration: 14.478403335s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.857378618s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:20:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.857405249s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXXX] duration: 5.001632103s trace: https://jaeger.dev.storj.io/trace/1e9d1a1ac3871c06 FAIL (2026-01-15 10:19:54 UTC) orbiter warning: context deadline exceeded Duration: 5.001651509s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.............X........................X...................] duration: 19.351495291s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:19:53 UTC) orbiter warning: context deadline exceeded Duration: 19.351523094s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 17.13387191s trace: https://jaeger.dev.storj.io/trace/6695e64327811e17 FAIL (2026-01-15 10:19:52 UTC) orbiter warning: context deadline exceeded Duration: 17.133895917s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.791730081s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:19:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.791760489s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XXX] duration: 5.001316566s trace: https://jaeger.dev.storj.io/trace/690a64e5ca7ec01 FAIL (2026-01-15 10:19:06 UTC) orbiter warning: context deadline exceeded Duration: 5.00134008s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.............X........................X..................] duration: 25.94514486s trace: https://jaeger.dev.storj.io/trace/1963fc7eec58afe5 FAIL (2026-01-15 10:19:05 UTC) orbiter warning: context deadline exceeded Duration: 25.945168004s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.770368107s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:19:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.770392022s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 15.582510676s trace: https://jaeger.dev.storj.io/trace/6695e64327811e17 FAIL (2026-01-15 10:18:52 UTC) orbiter warning: context deadline exceeded Duration: 15.582533631s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 20.710847231s trace: https://jaeger.dev.storj.io/trace/6695e64327811e17 FAIL (2026-01-15 10:18:23 UTC) orbiter warning: context deadline exceeded Duration: 20.710947865s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 56 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX..XXXX.XX] duration: 5.006546148s trace: https://jaeger.dev.storj.io/trace/6d870d4774307f58 FAIL (2026-01-15 10:18:03 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.006571277s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.838415293s trace: https://jaeger.dev.storj.io/trace/2fbfa88584da3097 FAIL (2026-01-15 10:17:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A) context deadline exceeded; piecestore: (Node ID: 12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, Piece ID: B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA) context deadline exceeded; piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ) context deadline exceeded; piecestore: (Node ID: 18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, Piece ID: RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA) context deadline exceeded; piecestore: (Node ID: 12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, Piece ID: EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: 66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA) context deadline exceeded; piecestore: (Node ID: 129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, Piece ID: 5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ) context deadline exceeded ======LOGS====== 15 09:57:10.584609208 (978h28m10.455692243s) [upload=856661]: Starting upload Jan 15 09:57:10.584665575 (978h28m10.45574861s) [upload=856661]: Done starting upload Jan 15 09:57:10.584821653 (978h28m10.455904688s) [upload=856661]: Uploading segments... Jan 15 09:57:10.584924799 (978h28m10.456007834s) [upload=856661]: Got next segment. Inline: false Jan 15 09:57:10.584964745 (978h28m10.45604778s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 15 09:57:10.584971267 (978h28m10.456054302s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 15 09:57:10.823621074 (978h28m10.694704119s) [seg_pos={0 0}, upload=856661]: Begin upload segment... Jan 15 09:57:10.823670158 (978h28m10.694753203s) [seg_pos={0 0}, upload=856661]: Done begin upload segment. Jan 15 09:57:10.823699083 (978h28m10.694782118s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.823728498 (978h28m10.694811533s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824206489 (978h28m10.695289524s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824518944 (978h28m10.695601979s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.824792395 (978h28m10.69587543s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825052099 (978h28m10.696135134s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825332132 (978h28m10.696415167s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825594191 (978h28m10.696677226s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.825905745 (978h28m10.69698878s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827370684 (978h28m10.698453729s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.827464774 (978h28m10.698547809s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.828525433 (978h28m10.699608468s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.829504497 (978h28m10.700587532s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.830537605 (978h28m10.70162064s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.831515727 (978h28m10.702598782s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:10.832686085 (978h28m10.70376912s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Uploading piece... Jan 15 09:57:12.678890664 (978h28m12.549973699s) [upload=856661]: Next returned nil segment Jan 15 09:57:12.678902616 (978h28m12.549985651s) [upload=856661]: Waiting for error group managing segments... Jan 15 09:57:12.979771084 (978h28m12.850854119s) [noise=true , address=145.79.227.3:4150 , piece_id=EGMCZUYCYT55OALUIMJYN4EDUF762ENPIABC7UDTXYGF5TSZC6HA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:12.998492020 (978h28m12.869575055s) [noise=true , address=145.79.227.2:4113 , piece_id=66VZ5QEXZGFAEDNVNX4UTUP7QXBS74IMBG7OZAHM75SCNFSVZUMA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.069046000 (978h28m12.940129045s) [noise=true , address=45.91.116.8:4048 , piece_id=QDUT6PSMMR6R33G25MJQENZJRMD57XPMHFTIY6MICMHM27PEM4BA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072324613 (978h28m12.943407658s) [noise=true , address=145.79.227.43:4159 , piece_id=B244GTPMHXNNCBGUSQI5C3IQGUUGR44ZWSLOP7RPKLGUPUDIMG3Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.072965623 (978h28m12.944048658s) [noise=true , address=145.79.227.39:4015 , piece_id=RD66CV6A3NFNAQ2MRH3PAPFJKQAMTPIJEVWM34NTIWV5OJY7O6RA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.077925036 (978h28m12.949008071s) [noise=true , address=45.91.116.13:4154 , piece_id=2R5SOUTPOQFMRJE5OGLDYZLAUDECDM3N4D56WGODUCKJKB63BK5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.150486269 (978h28m13.021569304s) [noise=true , address=45.91.116.14:4191 , piece_id=OO6YT34FGNXCGASDTBQM4KVYWIQFHUMM3W3QPDQVNXNUSWOSQNKQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.325459061 (978h28m13.196542096s) [noise=true , address=145.79.227.37:4566 , piece_id=RCVH46FBMOCDSETZG66HFPYOYIMANZBC6EFCH3CZ6CWMEJNIXRWQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.421974484 (978h28m13.293057519s) [noise=true , address=145.79.227.52:4446 , piece_id=RTXCHP2ZS2T6WIOYXTHTG7LAHTKSAA355OOCGH7MZOTVJHYKIZSQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.771569210 (978h28m13.642652245s) [noise=true , address=45.91.116.10:4081 , piece_id=LFCM5DUATGQ5VKHRUEXJ2JRYIHS34DDDF4KOTXPTYE3V2F25QI4Q, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.820446565 (978h28m13.691529601s) [noise=true , address=145.79.227.11:4730 , piece_id=YITEPNI2QVT75XTOSVLSL4FC6IKV6BJV7ETLRDN6FC5D3GH7DJBQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:13.914547340 (978h28m13.785630375s) [noise=true , address=145.79.227.47:4265 , piece_id=IOK6SKMQUXSRIWWG4SNA6UYLRS25MTZD27WLWWNVQY43PY5NPY5A, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085692355 (978h28m13.95677539s) [noise=true , address=145.79.227.5:4221 , piece_id=5AFP2N2TD4WXV7CNQJ3ML6F547DM244J2VEWKZYSI7UPFKH6NLEQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: <nil> Jan 15 09:57:14.085711951 (978h28m13.956794986s) [seg_pos={0 0}, upload=856661]: Segment reached optimal threshold of 13 pieces. Jan 15 09:57:14.085884360 (978h28m13.956967395s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.085890541 (978h28m13.956973576s) [noise=true , address=45.91.116.4:4939 , piece_id=F3PHQLF3ZBRNTHVZOH5P7S5FV734QDPBIKJDTZ5Z5P7FTF6VGWRA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086012454 (978h28m13.957095489s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f): protocol: expected piece hash; context canceled; EOF Jan 15 09:57:14.086017754 (978h28m13.957100789s) [noise=true , address=77.90.31.13:4629 , piece_id=PXAFO5PPWGWZA3CYCVJ4SRMTSFPM3R5CUVNOYBIDH2XP74KQ3WZQ, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086221943 (978h28m13.957304978s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41428->45.91.116.12:4225: use of closed network connection Jan 15 09:57:14.086243493 (978h28m13.957326548s) [noise=true , address=45.91.116.12:4225 , piece_id=QZBZ756GPNXFLGN6RBHBLITBLIXQJ2KL6FGCWBQTI5X5NKBPTSGA, seg_pos={0 0}, upload=856661]: Long tail cancellation due to optimal threshold Jan 15 09:57:14.086302506 (978h28m13.957385541s) [seg_pos={0 0}, upload=856661]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 15 09:57:14.086422625 (978h28m13.95750566s) [upload=856661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 15 09:57:14.086442232 (978h28m13.957525267s) [upload=856661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 15 09:57:14.366038024 (978h28m14.237121069s) [upload=856661]: Done uploading segments... Jan 15 09:57:14.366095694 (978h28m14.237178729s) [upload=856661]: Upload finished. err: <nil> Duration: 5.838432686s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)