Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 451367 reports (Page 389 of 9028)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.X.X.X...X..X.X....X.........X....X.....X.XX.XX..........X..] duration: 28.757815552s trace: https://jaeger.dev.storj.io/trace/34386d24f7900f24 FAIL (2026-01-11 14:07:19 UTC) orbiter warning: context deadline exceeded Duration: 28.757843424s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 13.14433058s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:07:18 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 13.144348315s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [XX.....X..............X......X..............................] duration: 6.727654994s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:07:10 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.727681294s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.791171707s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:07:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.791200562s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 53 out of 60 is failed (threshold=3) [XXXXXXXXXX.XXXXXXXXXX.XXX.XXXXXXXXXXX..X..XXXXXXXXXXXXXXXXXX] duration: 30.000869999s trace: https://jaeger.dev.storj.io/trace/24240cfa783fe875 FAIL (2026-01-11 14:06:51 UTC) orbiter warning: context deadline exceeded Duration: 30.000893404s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000765443s trace: https://jaeger.dev.storj.io/trace/1fa4f734e86eba9b FAIL (2026-01-11 14:06:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000789008s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [..X.X.X...X..X.X....X.........X....X.....X.XX.XX..........X.] duration: 46.591282477s trace: https://jaeger.dev.storj.io/trace/34386d24f7900f24 FAIL (2026-01-11 14:06:41 UTC) orbiter warning: context deadline exceeded Duration: 46.591296635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 4.797758102s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:06:35 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.798145999s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.XX.....X..............X......X.............................] duration: 11.61229404s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:06:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 11.612321792s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001286846s trace: https://jaeger.dev.storj.io/trace/2dcbd63a781e23c5 FAIL (2026-01-11 14:06:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001309668s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.667694448s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:06:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.667730736s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.734473254s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:05:59 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 2.734499764s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [...X.X.X...X..X.X....X.........X....X.....X.XX.XX..........X] duration: 1m0.001514897s trace: https://jaeger.dev.storj.io/trace/34386d24f7900f24 FAIL (2026-01-11 14:05:54 UTC) orbiter warning: context deadline exceeded Duration: 1m0.00153861s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 53 out of 60 is failed (threshold=3) [XXXXXXXXXXX.XXXXXXXXXX.XXX.XXXXXXXXXXX..X..XXXXXXXXXXXXXXXXX] duration: 30.001238139s trace: https://jaeger.dev.storj.io/trace/1e49a2e85ac7a909 FAIL (2026-01-11 14:05:41 UTC) orbiter warning: context deadline exceeded Duration: 30.001260362s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [..XX.....X..............X......X............................] duration: 7.546658129s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:05:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 7.546744362s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.768563126s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:05:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.768585899s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [...XX.....X..............X......X...........................] duration: 30.00045982s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:05:30 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 30.000487361s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001657403s trace: https://jaeger.dev.storj.io/trace/7a9117d665f00cc3 FAIL (2026-01-11 14:05:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001682911s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.199318756s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:05:25 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.199347661s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.785544779s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:05:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.785589383s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00124597s trace: https://jaeger.dev.storj.io/trace/45d8b6ecd3a3357b FAIL (2026-01-11 14:05:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00127193s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 8.163224389s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:04:58 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 8.163251199s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [....X.X.X...X..X.X....X.........X....X.....X.XX.XX..........] duration: 27.859468189s trace: https://jaeger.dev.storj.io/trace/126cf50c1c137347 FAIL (2026-01-11 14:04:50 UTC) orbiter warning: context deadline exceeded Duration: 27.859483048s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [....XX.....X..............X......X..........................] duration: 6.270175833s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:04:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.270208135s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.865236465s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:04:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.865319152s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 4.858372748s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:04:28 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.85840487s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 53 out of 60 is failed (threshold=3) [XXXXXXXXXXXX.XXXXXXXXXX.XXX.XXXXXXXXXXX..X..XXXXXXXXXXXXXXXX] duration: 30.001812339s trace: https://jaeger.dev.storj.io/trace/185d808bc71ec6b FAIL (2026-01-11 14:04:26 UTC) orbiter warning: context deadline exceeded Duration: 30.001838458s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....XX.....X..............X......X.........................] duration: 18.89991592s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:04:14 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.899928643s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.735298538s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:04:10 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 2.735322693s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [......XX.....X..............X......X........................] duration: 27.452554811s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:03:55 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 27.452585168s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000860456s trace: https://jaeger.dev.storj.io/trace/70d989b6c0590e71 FAIL (2026-01-11 14:03:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000880403s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [.....X.X.X...X..X.X....X.........X....X.....X.XX.XX.........] duration: 34.995959394s trace: https://jaeger.dev.storj.io/trace/126cf50c1c137347 FAIL (2026-01-11 14:03:43 UTC) orbiter warning: context deadline exceeded Duration: 34.995984221s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 4.970256229s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:03:42 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.970283291s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.911233181s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:03:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.911260483s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 13.210552153s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:03:26 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 13.210569818s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 53 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX.XXXXXXXXXX.XXX.XXXXXXXXXXX..X..XXXXXXXXXXXXXXX] duration: 30.000696519s trace: https://jaeger.dev.storj.io/trace/5133315a7df9aa04 FAIL (2026-01-11 14:03:22 UTC) orbiter warning: context deadline exceeded Duration: 30.000711347s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001056188s trace: https://jaeger.dev.storj.io/trace/174f2166e976be07 FAIL (2026-01-11 14:03:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001075145s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m0.000789225s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/70855e353f33f12b FAIL (2026-01-11 14:03:12 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 2m0.000820173s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 21.745442552s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:03:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 21.745472099s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [X.....X.X.X...X..X.X....X.........X....X.....X.XX.XX........] duration: 37.578524959s trace: https://jaeger.dev.storj.io/trace/126cf50c1c137347 FAIL (2026-01-11 14:02:59 UTC) orbiter warning: context deadline exceeded Duration: 37.578555196s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 4.784553024s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:02:54 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.784576169s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......XX.....X..............X......X.......................] duration: 8.557450373s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:02:47 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.557477293s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.243163482s trace: https://jaeger.dev.storj.io/trace/3e423ad27ef1ea01 FAIL (2026-01-11 14:02:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== an 11 14:01:56.920704205 (886h32m56.79178725s) [upload=775814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 11 14:01:56.920710487 (886h32m56.791793532s) [upload=775814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 11 14:01:57.167611308 (886h32m57.038694343s) [seg_pos={0 0}, upload=775814]: Begin upload segment... Jan 11 14:01:57.167653438 (886h32m57.038736473s) [seg_pos={0 0}, upload=775814]: Done begin upload segment. Jan 11 14:01:57.167676212 (886h32m57.038759247s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.167685169 (886h32m57.038768204s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168199689 (886h32m57.039282724s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168583339 (886h32m57.039666404s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.168909430 (886h32m57.039992465s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169194522 (886h32m57.040277557s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169438978 (886h32m57.040522013s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.169902650 (886h32m57.040985695s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170256034 (886h32m57.041339069s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170611110 (886h32m57.041694155s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.170913515 (886h32m57.04199655s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172535484 (886h32m57.043618529s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.172997033 (886h32m57.044080078s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.173560425 (886h32m57.04464347s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174116574 (886h32m57.045199609s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:57.174584827 (886h32m57.045667862s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Uploading piece... Jan 11 14:01:58.963709378 (886h32m58.834792413s) [upload=775814]: Next returned nil segment Jan 11 14:01:58.963721791 (886h32m58.834804836s) [upload=775814]: Waiting for error group managing segments... Jan 11 14:01:59.282421727 (886h32m59.153504762s) [noise=true , address=45.91.116.5:4300 , piece_id=SS2ZEA5SWP6V62JJMULAUH3C2FH4TD3YZ64M5M5LVV7K2QX4QMBA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:01:59.313031402 (886h32m59.184114437s) [noise=true , address=45.91.116.8:4046 , piece_id=ESNTDXUQPZX2HHI4PIPYZW3J55FCDQH5526DHHTWX7MQTMT7GYVA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.419452868 (886h33m0.290535903s) [noise=true , address=145.79.227.44:4194 , piece_id=PB6YOMUPMTMWHPAF23JSDMPMNGMDPPW2HKAGKW3UJUGJORNSUPXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.512030197 (886h33m0.383113232s) [noise=true , address=145.79.227.14:4837 , piece_id=XPFUWSMOIX2H24LN2FTDJKOY2ILSHXNB3ONYLD5SCQLA3NR3YOXA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:00.850966025 (886h33m0.72204906s) [noise=true , address=145.79.227.11:4729 , piece_id=ROO4EGXOZNBNEKHOWM526UCS3HT4KFJAGPFFUTRJNT76GZ6HATPA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.144755969 (886h33m1.015839014s) [noise=true , address=145.79.227.21:4473 , piece_id=TQBDMQBURHGHQGCRGBIVDWRQ52GX3LHM2GYSZ5DBGK7TQDA6XN3A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:01.908712660 (886h33m1.779795695s) [noise=true , address=145.79.227.7:4293 , piece_id=F5UTJTC3F55WGANUI4BFV2S6ZA3ZYM65SMG2KSQIXKPA43V6QZFA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:02.223235703 (886h33m2.094318738s) [noise=true , address=145.79.227.2:4114 , piece_id=ASG2XGEPBKIJKD5B3BZGX3KZG3INBQE6GNXIUR63KYWABWD773SQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:08.442466210 (886h33m8.313549255s) [noise=true , address=145.79.227.8:4329 , piece_id=UH4W242O5L25UCGKWSYIW3EOJMEV7K6LIHUPVIBSPZSFT2NPCDTQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: <nil> Jan 11 14:02:26.233089417 (886h33m26.104172452s) [noise=true , address=145.79.227.29:4942 , piece_id=DZEC32ACPW5LU4PTEYKUPAK7GC4TO2VOJAUG4ARJGKG6DL6F6TPQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55492->145.79.227.29:4942: use of closed network connection Jan 11 14:02:26.233664854 (886h33m26.104747889s) [noise=true , address=145.79.227.3:4150 , piece_id=VRBBPTE67PBYFWI2EJZQK2EVLLEJDQ2MVMP7FKU5G6KT3OUWFUOA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33892->145.79.227.3:4150: use of closed network connection Jan 11 14:02:26.233713606 (886h33m26.104796641s) [noise=true , address=145.79.227.18:4007 , piece_id=XDQ2367FB7ME22L2NS4V6AV7UAZ264O6C46L6RRC5X66IJNGVZMA, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:40372->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.233945338 (886h33m26.105028383s) [noise=true , address=145.79.227.17:4971 , piece_id=QZSH2UATLZAWRHUHYC7BGMI6ZPLQN7Z3KUMSUSTI2AU4ZVOEX42A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:34598->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234031070 (886h33m26.105114115s) [noise=true , address=145.79.227.51:4411 , piece_id=J5ZZCL4YZSXJH2KFL7XRXD4WQKM7MTX4XW54FRTAED2QDKMHQVUQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR, address:145.79.227.51:4411): protocol: read tcp 188.166.225.170:59004->145.79.227.51:4411: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234243455 (886h33m26.1053265s) [noise=true , address=145.79.227.32:4834 , piece_id=4AFNHEHSZRIS2IRCBMQDOT6NT5V56NDEJE7DJHCKI2FCYIVDO55A, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:59444->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234550900 (886h33m26.105633935s) [noise=true , address=145.79.227.37:4567 , piece_id=MMUEF6KRGYD67XMKCWVSO3TD3LCVJIZEULNGAXQCCNQ35KGL4OMQ, seg_pos={0 0}, upload=775814]: Done uploading piece. err: ecclient: upload failed (node:19uEj8XgtM7PdNP8KosTCF6H1dSrJ25x6fMshJHQbyrHzJ5sDS, address:145.79.227.37:4567): protocol: read tcp 188.166.225.170:60302->145.79.227.37:4567: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 11 14:02:26.234580246 (886h33m26.105663281s) [seg_pos={0 0}, upload=775814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jan 11 14:02:26.474589435 (886h33m26.34567249s) [upload=775814]: Done uploading segments... Jan 11 14:02:26.474630432 (886h33m26.345713478s) [upload=775814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.243187497s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 4.862251177s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:02:19 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 4.862279451s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001465345s trace: https://jaeger.dev.storj.io/trace/3c74d994214e8a8 FAIL (2026-01-11 14:02:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001488189s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 52 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXX.XXXXXXXXXX.XXX.XXXXXXXXXXX..X..XXXXXXXXXXXXXX] duration: 30.000956372s trace: https://jaeger.dev.storj.io/trace/7fd5eb54b8ffdbe3 FAIL (2026-01-11 14:02:11 UTC) orbiter warning: context deadline exceeded Duration: 30.000977333s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [........XX.....X..............X......X......................] duration: 15.247083117s trace: https://jaeger.dev.storj.io/trace/60a93ccca4497266 FAIL (2026-01-11 14:01:55 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 15.247107964s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.253353453s trace: https://jaeger.dev.storj.io/trace/75cc4fa5964171e9 FAIL (2026-01-11 14:01:42 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.253380062s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001457001s trace: https://jaeger.dev.storj.io/trace/2b1f3131e7b9b84a FAIL (2026-01-11 14:01:38 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001479563s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.X.....X.X.X...X..X.X....X.........X....X.....X.XX.XX.......] duration: 20.186503307s trace: https://jaeger.dev.storj.io/trace/126cf50c1c137347 FAIL (2026-01-11 14:01:37 UTC) orbiter warning: context deadline exceeded Duration: 20.186528334s core uplink us-select-1 us1-placement-20

1
...
387
388
389
390
391
...
9028