Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.X..X......................X..............................] duration: 16.288835334s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:14:16 UTC) orbiter warning: context deadline exceeded Duration: 16.28885486s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.X..X......................X.............................] duration: 25.480446632s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:13:45 UTC) orbiter warning: context deadline exceeded Duration: 25.480459566s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.........XX...X.X.............X.....X....X..................] duration: 5.770023335s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:13:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.770050197s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X..X.X..X......................X............................] duration: 17.771543624s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:13:20 UTC) orbiter warning: context deadline exceeded Duration: 17.773671977s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXX] duration: 5.001448551s trace: https://jaeger.dev.storj.io/trace/2ac75d78304c3a1a FAIL (2026-01-09 07:13:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001468009s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.........XX...X.X.............X.....X....X.................] duration: 5.837285212s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:13:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.837343385s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX] duration: 5.000705317s trace: https://jaeger.dev.storj.io/trace/19059f434b0d2870 FAIL (2026-01-09 07:12:29 UTC) orbiter warning: context deadline exceeded Duration: 5.000727227s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X..X.X..X......................X...........................] duration: 20.935832614s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:12:19 UTC) orbiter warning: context deadline exceeded Duration: 20.935854214s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X.........XX...X.X.............X.....X....X................] duration: 5.734364015s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:12:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.734389795s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXX] duration: 5.001559661s trace: https://jaeger.dev.storj.io/trace/7c3c4a2a7d0b3f07 FAIL (2026-01-09 07:12:04 UTC) orbiter warning: context deadline exceeded Duration: 5.001585349s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X..X.X..X......................X..........................] duration: 17.819632334s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:11:47 UTC) orbiter warning: context deadline exceeded Duration: 17.819656629s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXX] duration: 5.001228609s trace: https://jaeger.dev.storj.io/trace/6ce7b0999ca7ab23 FAIL (2026-01-09 07:11:23 UTC) orbiter warning: context deadline exceeded Duration: 5.001248648s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X..X.X..X......................X.........................] duration: 13.023549914s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:11:02 UTC) orbiter warning: context deadline exceeded Duration: 13.023590941s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X.........XX...X.X.............X.....X....X...............] duration: 5.809967321s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:10:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.809996746s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...X.........XX...X.X.............X.....X....X..............] duration: 5.827308199s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:10:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.827335119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....X.........XX...X.X.............X.....X....X.............] duration: 5.759750327s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:10:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.759779451s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX] duration: 5.00117528s trace: https://jaeger.dev.storj.io/trace/428ec6a6dbf64f9d FAIL (2026-01-09 07:10:10 UTC) orbiter warning: context deadline exceeded Duration: 5.001276293s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X..X.X..X......................X........................] duration: 8.10828026s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:09:59 UTC) orbiter warning: context deadline exceeded Duration: 8.108308584s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X] duration: 5.001591871s trace: https://jaeger.dev.storj.io/trace/7a38d5f0d83cd178 FAIL (2026-01-09 07:09:17 UTC) orbiter warning: context deadline exceeded Duration: 5.00160672s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....X.........XX...X.X.............X.....X....X............] duration: 5.773752306s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:09:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.773780289s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 58 out of 60 is failed (threshold=6) [XXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.] duration: 5.001346856s trace: https://jaeger.dev.storj.io/trace/6e5070ec3c075d4c FAIL (2026-01-09 07:08:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00136947s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..X.X..X......................X.......................] duration: 9.771135168s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:08:50 UTC) orbiter warning: context deadline exceeded Duration: 9.771166287s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001687539s trace: https://jaeger.dev.storj.io/trace/6e5070ec3c075d4c FAIL (2026-01-09 07:08:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001706535s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [......X.........XX...X.X.............X.....X....X...........] duration: 5.725423486s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:08:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.725453432s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.....X..X.X..X......................X......................] duration: 11.968499408s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:07:40 UTC) orbiter warning: context deadline exceeded Duration: 11.968526449s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00092319s trace: https://jaeger.dev.storj.io/trace/4be8ff805eca3c62 FAIL (2026-01-09 07:07:31 UTC) orbiter warning: context deadline exceeded Duration: 5.000941374s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001651086s trace: https://jaeger.dev.storj.io/trace/67a67b60d9e9c814 FAIL (2026-01-09 07:07:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001667819s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......X.........XX...X.X.............X.....X....X..........] duration: 5.835329495s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:06:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.8353585s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.....X..X.X..X......................X.....................] duration: 17.047446341s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:06:57 UTC) orbiter warning: context deadline exceeded Duration: 17.047575837s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X.....X..X.X..X......................X....................] duration: 20.289223551s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:06:33 UTC) orbiter warning: context deadline exceeded Duration: 20.289251784s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000613659s trace: https://jaeger.dev.storj.io/trace/2de23f0fa34eef FAIL (2026-01-09 07:06:16 UTC) orbiter warning: context deadline exceeded Duration: 5.000636973s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [........X.........XX...X.X.............X.....X....X.........] duration: 5.721129589s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:06:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.721155438s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X.....X..X.X..X......................X...................] duration: 15.064005148s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:05:56 UTC) orbiter warning: context deadline exceeded Duration: 15.064036848s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001905863s trace: https://jaeger.dev.storj.io/trace/6a20323f0732230f FAIL (2026-01-09 07:05:45 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001928876s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........X.........XX...X.X.............X.....X....X........] duration: 5.737646055s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:05:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.737673827s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.0013323s trace: https://jaeger.dev.storj.io/trace/e8eed4004b27c27 FAIL (2026-01-09 07:05:00 UTC) orbiter warning: context deadline exceeded Duration: 5.001362228s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X.....X..X.X..X......................X..................] duration: 19.125041131s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:04:50 UTC) orbiter warning: context deadline exceeded Duration: 19.125062602s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........X.........XX...X.X.............X.....X....X.......] duration: 5.837397284s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:04:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.837415168s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.....X..X.X..X......................X.................] duration: 12.482942795s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:04:04 UTC) orbiter warning: context deadline exceeded Duration: 12.482967993s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00069356s trace: https://jaeger.dev.storj.io/trace/7f644de4ef50079f FAIL (2026-01-09 07:03:59 UTC) orbiter warning: context deadline exceeded Duration: 5.000718666s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........X.........XX...X.X.............X.....X....X......] duration: 5.792601512s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:03:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.792635437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............X.........XX...X.X.............X.....X....X.....] duration: 5.708585441s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:03:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.708735145s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00108044s trace: https://jaeger.dev.storj.io/trace/13fe5ef0028c2ae2 FAIL (2026-01-09 07:03:00 UTC) orbiter warning: context deadline exceeded Duration: 5.001107341s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X.....X..X.X..X......................X................] duration: 14.92316291s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:02:55 UTC) orbiter warning: context deadline exceeded Duration: 14.923216151s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X.....X..X.X..X......................X...............] duration: 14.095039185s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:02:17 UTC) orbiter warning: context deadline exceeded Duration: 14.095065296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............X.........XX...X.X.............X.....X....X....] duration: 6.15014932s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:02:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 6.150179587s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X.....X..X.X..X......................X..............] duration: 22.422583281s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:01:56 UTC) orbiter warning: context deadline exceeded Duration: 22.422598169s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001641888s trace: https://jaeger.dev.storj.io/trace/11b31b9efb827a20 FAIL (2026-01-09 07:01:47 UTC) orbiter warning: context deadline exceeded Duration: 5.001665111s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.....X..X.X..X......................X.............] duration: 18.423504042s trace: https://jaeger.dev.storj.io/trace/4d4a7f7638a68428 FAIL (2026-01-09 07:01:03 UTC) orbiter warning: context deadline exceeded Duration: 18.423533108s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [X.............X.........XX...X.X.............X.....X....X...] duration: 5.852910967s trace: https://jaeger.dev.storj.io/trace/388fd50fe72a145e FAIL (2026-01-09 07:00:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== n 9 06:58:13.695272301 (831h29m13.566355346s) [upload=727286]: Starting upload Jan 9 06:58:13.695338286 (831h29m13.566421331s) [upload=727286]: Done starting upload Jan 9 06:58:13.695500856 (831h29m13.566583901s) [upload=727286]: Uploading segments... Jan 9 06:58:13.695769047 (831h29m13.566852082s) [upload=727286]: Got next segment. Inline: false Jan 9 06:58:13.695806127 (831h29m13.566889172s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 06:58:13.695812108 (831h29m13.566895144s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 06:58:13.945470344 (831h29m13.816553399s) [seg_pos={0 0}, upload=727286]: Begin upload segment... Jan 9 06:58:13.945521671 (831h29m13.816604706s) [seg_pos={0 0}, upload=727286]: Done begin upload segment. Jan 9 06:58:13.945559283 (831h29m13.816642328s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.945586395 (831h29m13.81666944s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946241853 (831h29m13.817324888s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.946662263 (831h29m13.817745308s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947149431 (831h29m13.818232486s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947586003 (831h29m13.818669048s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.947992246 (831h29m13.819075281s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948221212 (831h29m13.819304257s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948461410 (831h29m13.819544455s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948719861 (831h29m13.819802896s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.948898321 (831h29m13.819981356s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949200116 (831h29m13.820283161s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949314053 (831h29m13.820397098s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.949642358 (831h29m13.820725393s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950153292 (831h29m13.821236327s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:13.950643355 (831h29m13.8217264s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Uploading piece... Jan 9 06:58:15.655797138 (831h29m15.526880173s) [upload=727286]: Next returned nil segment Jan 9 06:58:15.655808910 (831h29m15.526891955s) [upload=727286]: Waiting for error group managing segments... Jan 9 06:58:15.936492946 (831h29m15.807575991s) [noise=true , address=145.79.227.3:4149 , piece_id=G6QOG3PIUO3TEZ7MUY33CKCVAQKNXQD2FJJXBFQS23U6VFFO2WNQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938126647 (831h29m15.809209692s) [noise=true , address=145.79.227.52:4446 , piece_id=6NGP26FGJWFVSQUKHFGMNYXK36Q2GKE3QMRMUAHMGBDIJTKRM42Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.938190048 (831h29m15.809273083s) [noise=true , address=145.79.227.37:4567 , piece_id=V5IFJPYOYADFTLQM4U462SDIR5YONXCRX5C5TECZRIEPQ5AGE7AQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941535648 (831h29m15.812618693s) [noise=true , address=145.79.227.11:4729 , piece_id=KITZ2ZCDYWYHL3V5XVTXX6LTD2JNYVSUXSSQBO7DJWCXNY6TQJ5Q, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.941599800 (831h29m15.812682835s) [noise=true , address=145.79.227.15:4876 , piece_id=IWQFL455354TREJSBNHBAE2N7D6FX34NXTFLLZ7ZTHTVLT4Q4O2A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950577376 (831h29m15.821660411s) [noise=true , address=145.79.227.7:4294 , piece_id=H72VIEACSLIG7BDQHJ3PZFEBLHFLC25MKCZNH5T3R2HISVTWWVOQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.950819457 (831h29m15.821902492s) [noise=true , address=145.79.227.50:4374 , piece_id=JDU2FNTABG2VIJIIRTZ5LDQVRXWUI44LXEP7IHTBMHMGSEQXCL6A, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.958061838 (831h29m15.829144883s) [noise=true , address=145.79.227.14:4837 , piece_id=XDLL3OXLGLQOW4ZRSSOIINCSSLN7ABAOGRAJF3ZA42NEKPKCKRWA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:15.959409655 (831h29m15.83049269s) [noise=true , address=145.79.227.35:4481 , piece_id=7GUTZCM7CONEZFZQ54OH4BD54LQSO6CYQEM3YRBAJYGRY2JCNUPQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.044492585 (831h29m15.91557562s) [noise=true , address=145.79.227.38:4977 , piece_id=BWDZ4FPKUXKMSU5YESRQ6CDJNYRBELY37Z2Y3YOR6EYZ3KGCGTZA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.238220553 (831h29m16.109303599s) [noise=true , address=77.90.31.13:4629 , piece_id=RFCRJEEIIDGDMQX3F26EYRKVYUZCAMB4KGEPKP6WAX2WYT55OUQQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.245229681 (831h29m16.116312726s) [noise=true , address=77.90.31.15:4697 , piece_id=LGDDNKCQLIHDQ6NKPGH2PDKVOVDE3VJXQUSZMKSKDEXQ4AMT72YQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357051368 (831h29m16.228134413s) [noise=true , address=77.90.31.14:4661 , piece_id=6BVCVUWUXP6CY4O6K5FTDEG4P6QZSJYKKU2DOJ7KTYQEPF2ZROZQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: <nil> Jan 9 06:58:16.357077027 (831h29m16.228160062s) [seg_pos={0 0}, upload=727286]: Segment reached optimal threshold of 13 pieces. Jan 9 06:58:16.357309239 (831h29m16.228392284s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357318416 (831h29m16.228401462s) [noise=true , address=45.91.116.5:4297 , piece_id=AB42Q7YQNKXP72WA7X47Z4FJ4BQF4FWCTBUZX264XFW6XMYFDEMQ, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357581948 (831h29m16.228664993s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jan 9 06:58:16.357607006 (831h29m16.228690061s) [noise=true , address=45.91.116.10:4083 , piece_id=E3Y3AJDRWXKAMGFS2PT32BQ564RYIUNEFNP636VUFA5NIH5LERYA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357638056 (831h29m16.228721111s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46782->45.91.116.8:4047: use of closed network connection Jan 9 06:58:16.357650178 (831h29m16.228733233s) [noise=true , address=45.91.116.8:4047 , piece_id=7TZK3LF4VJBIEEHL47WWQAYP562DLROMGHH3QTXLVU2EW7XB6TDA, seg_pos={0 0}, upload=727286]: Long tail cancellation due to optimal threshold Jan 9 06:58:16.357673874 (831h29m16.228756909s) [seg_pos={0 0}, upload=727286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 06:58:16.357759596 (831h29m16.228842652s) [upload=727286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 06:58:16.357766750 (831h29m16.228849805s) [upload=727286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 06:58:16.644899615 (831h29m16.51598265s) [upload=727286]: Done uploading segments... Jan 9 06:58:16.644920265 (831h29m16.51600331s) [upload=727286]: Upload finished. err: <nil> Duration: 5.852935613s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)