Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.484861521s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:40:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 6.484887801s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.823958981s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:40:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.823989761s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
5.8727317s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:40:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.872759513s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.....................................................XX.X.X.] duration:
3.57185969s trace:
https://jaeger.dev.storj.io/trace/5366ea4092e184b8
FAIL (2026-04-11 00:40:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.571883085s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.513370871s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:40:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 6.513397462s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
5.726272262s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:39:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.726293892s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.807296634s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:39:53 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.807318756s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.623150341s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:39:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 6.623173285s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [......................................................XX.X.X] duration:
5.002034468s trace:
https://jaeger.dev.storj.io/trace/5366ea4092e184b8
FAIL (2026-04-11 00:39:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00205636s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
5.737485583s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:39:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.737508938s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
26.976713604s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:39:18 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 26.976753791s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
5.764846775s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:39:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.764883185s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
6.500435751s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:38:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 6.500460699s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.......................................................XX.X.] duration:
3.520134914s trace:
https://jaeger.dev.storj.io/trace/4d0062f369162a92
FAIL (2026-04-11 00:38:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.520162387s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.819006915s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:38:30 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.819019751s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
6.460825967s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:38:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 6.460874041s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
5.776810985s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:37:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.77683325s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [........................................................XX.X] duration:
5.000451864s trace:
https://jaeger.dev.storj.io/trace/4d0062f369162a92
FAIL (2026-04-11 00:37:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000474608s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
5.708172651s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:37:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 5.70819327s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
15.344298256s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:37:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.344318213s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.873673498s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:37:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 5.873699239s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
8.304610348s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:36:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 8.304630768s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................................................XX.] duration:
4.476818785s trace:
https://jaeger.dev.storj.io/trace/26a985775675b83e
FAIL (2026-04-11 00:36:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.476836579s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
18.005412656s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:36:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.005436793s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
16.885208192s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:36:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 16.885227869s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
23.058370755s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:36:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 23.058386796s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.281430055s trace:
https://jaeger.dev.storj.io/trace/7bf7d700005c5ffc
FAIL (2026-04-11 00:36:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
1 00:35:02.102980102 (80h48m48.922216136s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Apr 11 00:35:02.102988498 (80h48m48.922224532s) [upload=77643]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:35:02.347014884 (80h48m49.166250918s) [seg_pos={0 0}, upload=77643]: Begin upload segment...
Apr 11 00:35:02.347096521 (80h48m49.166332555s) [seg_pos={0 0}, upload=77643]: Done begin upload segment.
Apr 11 00:35:02.347134595 (80h48m49.166370629s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347219359 (80h48m49.166455393s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347397994 (80h48m49.166634028s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.347951241 (80h48m49.167187295s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.348403333 (80h48m49.167639377s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.349580416 (80h48m49.16881644s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350222825 (80h48m49.169458859s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.350716198 (80h48m49.169952232s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351232323 (80h48m49.170468358s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.351725215 (80h48m49.170961259s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352236692 (80h48m49.171472726s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.352725695 (80h48m49.171961729s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.353282871 (80h48m49.172518905s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.355150295 (80h48m49.174386329s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356256440 (80h48m49.175492474s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.356714043 (80h48m49.175950077s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.358291207 (80h48m49.177527241s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:02.360085730 (80h48m49.179321764s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Uploading piece...
Apr 11 00:35:03.644395589 (80h48m50.463631623s) [upload=77643]: Next returned nil segment
Apr 11 00:35:03.644419985 (80h48m50.463656019s) [upload=77643]: Waiting for error group managing segments...
Apr 11 00:35:04.076514972 (80h48m50.895751006s) [noise=true , address=193.149.164.71:4296 , piece_id=ICYYOFCZH7TADQFI2TGSPCEJXX4WZELENPWM62K2ZMF2K2YO566Q, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087396093 (80h48m50.906632127s) [noise=true , address=193.149.164.70:4132 , piece_id=CGIAUTQFH6QFLO5NQBTRFUKEJP4V35ERVC5YMWIM7TFF7U7CWOSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.087802908 (80h48m50.907038942s) [noise=true , address=64.49.13.26:4948 , piece_id=6YCV7WXCWU47K7SWDZPRMRMQQGDPEDUJSPGE2N2JATBLHRHJ42UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.101188451 (80h48m50.920424475s) [noise=true , address=64.49.13.137:4503 , piece_id=52NNEHEWZWCQWFIFDX4DFYA4MWMRRASOLYLHDPXXT3GVRQT5S4UA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.129414164 (80h48m50.948650208s) [noise=true , address=193.149.164.67:4007 , piece_id=XJSSKWPSBIUORDJ2OC64EOK6JOBLIXSP7MQYKSGGJQNLW7NSAH3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.130587098 (80h48m50.949823132s) [noise=true , address=193.149.164.68:4055 , piece_id=7HP27JRNWZEBXVPI6I4BV5YBBUH2WM5CH2MEFAMWY5GQ6FQSRBOQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.145662963 (80h48m50.964898997s) [noise=true , address=64.49.13.170:4719 , piece_id=RSUNMVCLMQH7DIECFJCHZXMM4NOK3RJXDMK66B7GX2KTZ2W6SGXQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.148310473 (80h48m50.967546507s) [noise=true , address=193.149.164.69:4069 , piece_id=FUIABW2DEM66WHOM3FDNFXOVQUOVFMEBXRXGZFGYDFMHY2SCJ2KQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.253409518 (80h48m51.072645562s) [noise=true , address=45.91.116.13:4186 , piece_id=UJJE5F7N74ME5E7OBBXTA5KEPMFNY4WZ4AHSOX4HRNWYFNBPYBSA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.320968782 (80h48m51.140204816s) [noise=true , address=77.90.31.15:4723 , piece_id=Z4EC4PUVABSBPPK56JR3IB4ULAAYOXIPLHTWGNJEMBLQRX4KQRTA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:04.385096245 (80h48m51.204332289s) [noise=true , address=45.91.116.7:4028 , piece_id=A3JVIG2WKBOAVQJXWET3YCUOD5WGIGCYRQYHJDZOMABMUA6RXWHQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:35:05.729858372 (80h48m52.549094406s) [noise=true , address=77.90.31.17:4787 , piece_id=BM62FJFUV3OICUAGDRZL4SVDWJBVRFXXBMSCWVXGK2MZEOFJ5GIQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: <nil>
Apr 11 00:36:01.437011024 (80h49m48.256247048s) [noise=true , address=145.79.227.37:4584 , piece_id=PJHMI7QYRI3A7WZAEUM2ONTS4IHN62QBRL6MAFWWYGJHP3DBNN3A, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12uS1kJzcCiNR4GwBxZqkGoSoGYJCDnwUPMpJLn6KrPE59Hq2S, address:145.79.227.37:4584): protocol: rpc: dial tcp 145.79.227.37:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437062985 (80h49m48.256299009s) [noise=true , address=145.79.227.33:4875 , piece_id=Q6WMCYRBOLPPTLTRWLLRNFDHGTY2NE3J3KEYGI6U4YEMXB2XKCFQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1, address:145.79.227.33:4875): protocol: rpc: dial tcp 145.79.227.33:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437204669 (80h49m48.256440693s) [noise=true , address=145.79.227.46:4625 , piece_id=CDFOLHPDRI4DVXHEHB46WWUSR3VRLYGVPQIQO7N2FMONQ2QNPELQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1qZcuemXhaKWsit4JtJ9MoYjwECsHviNrdAzTbWrPQADn4RbGP, address:145.79.227.46:4625): protocol: rpc: dial tcp 145.79.227.46:4625: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437311625 (80h49m48.256547649s) [noise=true , address=145.79.227.13:4824 , piece_id=2FRAED3YTIE65VE72MLIUFDU72JBTHL27QDRE7SK6RIO7PBKYWBA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:145.79.227.13:4824): protocol: rpc: dial tcp 145.79.227.13:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437529275 (80h49m48.256765299s) [noise=true , address=145.79.227.43:4169 , piece_id=U3B2KD66A5Z4SUJAKBYZWUQEJBCGIVGDDAKT74XBCXPVDN3DMQHA, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:1eSyfixGcCi7TCFscr473WQvg7ujGfcr33pUdGN4dRngroiN5i, address:145.79.227.43:4169): protocol: rpc: dial tcp 145.79.227.43:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437574683 (80h49m48.256810717s) [noise=true , address=145.79.227.35:4490 , piece_id=QWHI2B4W3HDXOI2JHHQSQ2BD2CMRNDCASZNSXUQNTOUKI4VZB6VQ, seg_pos={0 0}, upload=77643]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:145.79.227.35:4490): protocol: rpc: dial tcp 145.79.227.35:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:01.437616122 (80h49m48.256852156s) [seg_pos={0 0}, upload=77643]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Apr 11 00:36:01.715269026 (80h49m48.53450508s) [upload=77643]: Done uploading segments...
Apr 11 00:36:01.715313421 (80h49m48.534549455s) [upload=77643]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.281456487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
36.844531725s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:36:00 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 36.844543297s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.232944563s trace:
https://jaeger.dev.storj.io/trace/276b2e32db11ea61
FAIL (2026-04-11 00:35:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 1)
======LOGS======
BBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446530549 (80h48m57.265766573s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.446959266 (80h48m57.26619529s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447310023 (80h48m57.266546047s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447634449 (80h48m57.266870473s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.447903609 (80h48m57.267139633s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448200061 (80h48m57.267436085s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.448524397 (80h48m57.267760431s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.449987020 (80h48m57.269223044s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.450280857 (80h48m57.269516882s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.451699786 (80h48m57.27093581s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.452991929 (80h48m57.272227953s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.454275917 (80h48m57.273511941s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.455479750 (80h48m57.274715775s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:10.456609872 (80h48m57.275845906s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Uploading piece...
Apr 11 00:35:12.280983620 (80h48m59.100219644s) [upload=77646]: Next returned nil segment
Apr 11 00:35:12.280999701 (80h48m59.100235735s) [upload=77646]: Waiting for error group managing segments...
Apr 11 00:35:14.470144773 (80h49m1.289380797s) [noise=true , address=77.90.31.17:4769 , piece_id=KSZRPYQ4245Q5HHYIGREJGRQFFIX7II3G25VQL3QUKGMQACUX4PQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: <nil>
Apr 11 00:35:39.550189016 (80h49m26.36942505s) [noise=true , address=145.79.227.41:4087 , piece_id=UGSA2KGDKKDUKH56ZRMGWBFIE3GVJSVSXJM7J5UOXTVD3Z7BXLWQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1BfeGD2MGRPqsBRGLioTiTAcPykpSTUUx2CBVSPQcz4LQqFdWR, address:145.79.227.41:4087): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550223232 (80h49m26.369459256s) [noise=true , address=145.79.227.27:4692 , piece_id=NRPL7ABNY5VOBM3CCV564NOJUYBVEVS6RP7UKSOYCJ3E2QJ55IUA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:145.79.227.27:4692): protocol: rpc: dial tcp 145.79.227.27:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550359355 (80h49m26.369595379s) [noise=true , address=145.79.227.13:4804 , piece_id=GUJNCXOBUHKIFXVKUIGLOWKBTNEHSRPT2W4BXJDENUOH5TA6JFNA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550423339 (80h49m26.369659363s) [noise=true , address=145.79.227.10:4407 , piece_id=R6QCBNR6SDAL3Z6OV5BW5IDH3WPVEVNRITQYOP27QAYZFPEKXDDQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: rpc: dial tcp 145.79.227.10:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550507481 (80h49m26.369743505s) [noise=true , address=145.79.227.26:4653 , piece_id=UXV2X7X5NCDJCF2KWMFSZYJ4OL3WPZTK2SBXIAIGXPCKNE4SMJ3Q, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550602244 (80h49m26.369838268s) [noise=true , address=145.79.227.4:4189 , piece_id=X6FDGNOFONSJ2NX6Y5BQ66M2YFPB4G626AHD3IWIYVMLH33ZO3UA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550702908 (80h49m26.369938932s) [noise=true , address=145.79.227.8:4332 , piece_id=F2OTPJR2GQW3DGMPS4MILL7GJ7TXJ7YFUFBBWVUOB2MYAYVTHDAA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.550756361 (80h49m26.369992385s) [noise=true , address=145.79.227.39:4015 , piece_id=MGIMNB3YUJLI6OI5EBZOD655WNRLUZC3YY52ZTA6SB2DAFSZZ2TQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550854590 (80h49m26.370090614s) [noise=true , address=145.79.227.28:4729 , piece_id=YSYFJAPUJUMBHWNJN4EAITQ4HYQ2BUGGHM6UMY6X34GQT3JWC6QQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12ruoUvv6Q9UoRJwugnxbUREY1rjuCipHFtteYbwg5HYCHDXQ3L, address:145.79.227.28:4729): protocol: rpc: dial tcp 145.79.227.28:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550897464 (80h49m26.370133498s) [noise=true , address=145.79.227.52:4445 , piece_id=UESFHZZHC4APJUVTEQNFS6W7SUAUMIWOM5IARKF6UGDGVXQZ3EYA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550956588 (80h49m26.370192612s) [noise=true , address=145.79.227.12:4770 , piece_id=SCPQOYZQV2NAYYYTUW64J36FYH5SLCVBS4ZME6OLFCPM72UALPBQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:145.79.227.12:4770): protocol: rpc: dial tcp 145.79.227.12:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.550985494 (80h49m26.370221518s) [noise=true , address=145.79.227.15:4873 , piece_id=XW5BM3OMNITH5HL2S6EUADYD2ZIW4AHZGROMBZCIX6TF7IVFOK6A, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:39.551000572 (80h49m26.370236596s) [noise=true , address=145.79.227.14:4839 , piece_id=QBQZJPH3ND4CFJXFOPLBI2W3OJ75DRNCWYBIF6N3Z757BEEBJJEA, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551084434 (80h49m26.370320469s) [noise=true , address=145.79.227.22:4513 , piece_id=ADDMI4UFZULT4KYF26R7R37LBG5IELKP57UY5LLB7QTL4DSSKFLQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551180850 (80h49m26.370416874s) [noise=true , address=145.79.227.20:4441 , piece_id=VWOXNGMUDFI5XKTTLBNMXOXXIDD4PGABJOXUIMG2QAX4ISNJP2CQ, seg_pos={0 0}, upload=77646]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:39.551213854 (80h49m26.370449878s) [seg_pos={0 0}, upload=77646]: Done waiting for segment. successful: 1 optimal: 13 errs: <nil>
Apr 11 00:35:39.781642337 (80h49m26.600878371s) [upload=77646]: Done uploading segments...
Apr 11 00:35:39.781678407 (80h49m26.600914441s) [upload=77646]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 1)
Duration: 30.232964801s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........................................................XX] duration:
5.001034881s trace:
https://jaeger.dev.storj.io/trace/26a985775675b83e
FAIL (2026-04-11 00:35:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001079946s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.000832251s trace:
https://jaeger.dev.storj.io/trace/2ead608de6f962d2
FAIL (2026-04-11 00:35:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000853944s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.248572592s trace:
https://jaeger.dev.storj.io/trace/6a38415320a171ec
FAIL (2026-04-11 00:35:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
9:4016 , piece_id=OXVS5AXJRMFZQUCEJ7VETTYOB3JKAFFDENJGSVMRGL2VDL4ZJXJA, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.395389282 (80h48m25.214625307s) [noise=true , address=145.79.227.22:4510 , piece_id=3JJRA73X5YC4CE2KOEKX2YQ4EM6456IDWS22YCXPGNBSQPQKNGFQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.395464568 (80h48m25.214700592s) [noise=true , address=145.79.227.10:4406 , piece_id=4G5IEVWBTVB7LDOP3GFMLFDU3FNHT6K42QYYCBB7G4IB4CUJXF3A, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.395792471 (80h48m25.215028495s) [noise=true , address=145.79.227.52:4447 , piece_id=MFETPLXO3HYSJSRXTR47NMJ2UDWVK7P5F2HKYMQIJKTWO3YXKWSQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.396066029 (80h48m25.215302053s) [noise=true , address=145.79.227.9:4371 , piece_id=FT4LGDL475KSZJO5RRPZOQ2TTJE3URAJ6MPBWVF5OYGJ3PGK22JA, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.396353893 (80h48m25.215589918s) [noise=true , address=145.79.227.45:4235 , piece_id=XXDZ4ZM3IKFYQGBSQBKP42O7MAM4QTBR6LQGMENMO4YPGN2YW7VQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.396660996 (80h48m25.21589701s) [noise=true , address=145.79.227.21:4478 , piece_id=OIUYQE5E4OB7SQTA5LXQ57YWP42WPEKYZ4WNHXNOJTCG5PS44HLQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.396728066 (80h48m25.21596409s) [noise=true , address=145.79.227.19:4082 , piece_id=BHZJ4RTTFK2B3SCCIHUCQO5ZVDQNJBSLMZYF54YLNSAJ3I4CAVKA, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.396886722 (80h48m25.216122746s) [noise=true , address=77.90.31.10:4518 , piece_id=3LA6XMCAEZCKYLZPXJJXPZTKKBLYVEM5PKYV7LHB7WHSVRS544JA, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.397116125 (80h48m25.216352149s) [noise=true , address=145.79.227.25:4620 , piece_id=GGXJHUJIUAAS4P55SCVNPZ3ZMGOSWX4EAAOUXEWCQMBGJGUVSNKQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.397337713 (80h48m25.216573737s) [noise=true , address=77.90.31.18:4805 , piece_id=L3PTNXKBDIOAS3ZPTX2CB4RBWLJD2VTT3FRDLJXGWWO2APGL4QTA, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.397568849 (80h48m25.216804873s) [noise=true , address=145.79.227.4:4190 , piece_id=SLNFOLHOVE2OMC22LHZLH5I5JNRYZ3U2O5FUH7BA6GI57XY2QTEQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.397792711 (80h48m25.217028735s) [noise=true , address=145.79.227.23:4551 , piece_id=GRXLAPLNEQBN7NTL4VCFOOOBDU3EJYEOQXL2XQO2KHPEYXWRKMFQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.398100535 (80h48m25.217336559s) [noise=true , address=145.79.227.20:4442 , piece_id=S4XMMHJYALYDMATQ4VVWAXPILENIRPLTMWLFEP4RPMSYDEPUNWKQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:38.398716864 (80h48m25.217952888s) [noise=true , address=77.90.31.13:4627 , piece_id=LEH4XVP64ELZN5GH4FSGJEUVLRWY4FQVZIXIR2N333C5DQ3NIPVQ, seg_pos={0 0}, upload=77636]: Uploading piece...
Apr 11 00:34:40.196669560 (80h48m27.015905584s) [upload=77636]: Next returned nil segment
Apr 11 00:34:40.196681353 (80h48m27.015917377s) [upload=77636]: Waiting for error group managing segments...
Apr 11 00:34:40.484988978 (80h48m27.304225012s) [noise=true , address=77.90.31.10:4518 , piece_id=3LA6XMCAEZCKYLZPXJJXPZTKKBLYVEM5PKYV7LHB7WHSVRS544JA, seg_pos={0 0}, upload=77636]: Done uploading piece. err: <nil>
Apr 11 00:34:40.514150598 (80h48m27.333386662s) [noise=true , address=77.90.31.13:4627 , piece_id=LEH4XVP64ELZN5GH4FSGJEUVLRWY4FQVZIXIR2N333C5DQ3NIPVQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: <nil>
Apr 11 00:34:40.527014376 (80h48m27.34625043s) [noise=true , address=77.90.31.18:4805 , piece_id=L3PTNXKBDIOAS3ZPTX2CB4RBWLJD2VTT3FRDLJXGWWO2APGL4QTA, seg_pos={0 0}, upload=77636]: Done uploading piece. err: <nil>
Apr 11 00:35:07.474601920 (80h48m54.293837944s) [noise=true , address=145.79.227.22:4510 , piece_id=3JJRA73X5YC4CE2KOEKX2YQ4EM6456IDWS22YCXPGNBSQPQKNGFQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.474797508 (80h48m54.294033532s) [noise=true , address=145.79.227.9:4371 , piece_id=FT4LGDL475KSZJO5RRPZOQ2TTJE3URAJ6MPBWVF5OYGJ3PGK22JA, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12cSqcGZqvZqs1QKBi6WK4fUDKgvQcmTupR1C9ZkWz879XeWPEL, address:145.79.227.9:4371): protocol: rpc: dial tcp 145.79.227.9:4371: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.474819169 (80h48m54.294055193s) [noise=true , address=145.79.227.27:4689 , piece_id=BL6GKRRDZ4NNTL2MR5VT7CPIJH56TMX27CC7AG2SQUY7F6ZMBG4Q, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: rpc: dial tcp 145.79.227.27:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.474961684 (80h48m54.294197709s) [noise=true , address=145.79.227.21:4478 , piece_id=OIUYQE5E4OB7SQTA5LXQ57YWP42WPEKYZ4WNHXNOJTCG5PS44HLQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:17kFjjJyvHApPUxUHJ6HeMEBCoo6xTbr8QizXjBEHwWj3mZUha, address:145.79.227.21:4478): protocol: rpc: dial tcp 145.79.227.21:4478: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475226385 (80h48m54.294462409s) [noise=true , address=145.79.227.52:4447 , piece_id=MFETPLXO3HYSJSRXTR47NMJ2UDWVK7P5F2HKYMQIJKTWO3YXKWSQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475246114 (80h48m54.294482138s) [noise=true , address=145.79.227.23:4551 , piece_id=GRXLAPLNEQBN7NTL4VCFOOOBDU3EJYEOQXL2XQO2KHPEYXWRKMFQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:145.79.227.23:4551): protocol: rpc: dial tcp 145.79.227.23:4551: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475399870 (80h48m54.294635894s) [noise=true , address=145.79.227.20:4442 , piece_id=S4XMMHJYALYDMATQ4VVWAXPILENIRPLTMWLFEP4RPMSYDEPUNWKQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:125KyoqeSzbjP8XPhXqw2AenTDiLtVmt5KpLBD7ZW7YC9e9zJu9, address:145.79.227.20:4442): protocol: rpc: dial tcp 145.79.227.20:4442: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475547946 (80h48m54.29478398s) [noise=true , address=145.79.227.45:4235 , piece_id=XXDZ4ZM3IKFYQGBSQBKP42O7MAM4QTBR6LQGMENMO4YPGN2YW7VQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT, address:145.79.227.45:4235): protocol: rpc: dial tcp 145.79.227.45:4235: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475563666 (80h48m54.294800151s) [noise=true , address=145.79.227.19:4082 , piece_id=BHZJ4RTTFK2B3SCCIHUCQO5ZVDQNJBSLMZYF54YLNSAJ3I4CAVKA, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12DKsNCtqxLvpJwLszdLs74V3eb8NH2VAV9V3bngzUczawmNoLK, address:145.79.227.19:4082): protocol: rpc: dial tcp 145.79.227.19:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475681784 (80h48m54.294917808s) [noise=true , address=145.79.227.25:4620 , piece_id=GGXJHUJIUAAS4P55SCVNPZ3ZMGOSWX4EAAOUXEWCQMBGJGUVSNKQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475805313 (80h48m54.295041347s) [noise=true , address=145.79.227.4:4190 , piece_id=SLNFOLHOVE2OMC22LHZLH5I5JNRYZ3U2O5FUH7BA6GI57XY2QTEQ, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:1eNJzs5Dc7Nv7imoQXZDp2mEVWjD8o4viGVkFvuAeZ1arynxu4, address:145.79.227.4:4190): protocol: rpc: dial tcp 145.79.227.4:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.475967937 (80h48m54.295203961s) [noise=true , address=145.79.227.39:4016 , piece_id=OXVS5AXJRMFZQUCEJ7VETTYOB3JKAFFDENJGSVMRGL2VDL4ZJXJA, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:1jC98jEWn6K8jFHPA9knJeHAkuNEmcS7ThxUvbySDfbq6FmKVR, address:145.79.227.39:4016): protocol: rpc: dial tcp 145.79.227.39:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.476160638 (80h48m54.295396662s) [noise=true , address=145.79.227.10:4406 , piece_id=4G5IEVWBTVB7LDOP3GFMLFDU3FNHT6K42QYYCBB7G4IB4CUJXF3A, seg_pos={0 0}, upload=77636]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:07.476181249 (80h48m54.295417273s) [seg_pos={0 0}, upload=77636]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:35:07.722037726 (80h48m54.54127375s) [upload=77636]: Done uploading segments...
Apr 11 00:35:07.722082142 (80h48m54.541318166s) [upload=77636]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 30.248592511s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001120425s trace:
https://jaeger.dev.storj.io/trace/22e654e1e343d432
FAIL (2026-04-11 00:35:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001146305s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
3.569305467s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:42:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.569340073s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
3.518759751s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:41:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.518796051s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
3.493405223s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:40:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.493434048s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
3.48045915s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:39:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.480483166s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
3.62319966s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:38:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.623338759s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
3.465865545s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:37:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.465886844s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
3.362681776s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:36:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.36270478s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
3.456041455s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:36:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.45606468s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
3.524060224s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:35:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.524081065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
3.483883979s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:34:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.483904739s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
3.895462412s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:33:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.895499032s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
3.531108085s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:33:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.531130939s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............X...............................................] duration:
3.525128587s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:33:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.525159076s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............X..............................................] duration:
3.51639914s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:32:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.516423377s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............X.............................................] duration:
4.425103656s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:30:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.425124987s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............X............................................] duration:
3.548861298s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:30:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.548873924s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................X...........................................] duration:
3.53741962s trace:
https://jaeger.dev.storj.io/trace/5971dbab6579e325
FAIL (2026-04-10 22:29:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.537443165s
core
uplink
us-select-1
us1-placement-20
|