Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 11.20881395s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:28:58 UTC) orbiter warning: context deadline exceeded Duration: 11.208833307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.716384747s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:28:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.716412842s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 26 out of 60 is failed (threshold=6) [..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....X.XX.] duration: 3.703473111s trace: https://jaeger.dev.storj.io/trace/3302eb86b1a8da4c FAIL (2026-01-14 19:28:17 UTC) orbiter warning: context deadline exceeded Duration: 3.704035051s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 10.104364705s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:27:58 UTC) orbiter warning: context deadline exceeded Duration: 10.104392266s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 9.062060574s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:27:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.062090001s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.49335772s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:27:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.493386263s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [X..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....X.XX] duration: 5.001433829s trace: https://jaeger.dev.storj.io/trace/3302eb86b1a8da4c FAIL (2026-01-14 19:27:03 UTC) orbiter warning: context deadline exceeded Duration: 5.001457313s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.758437553s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:26:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.758460647s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 12.925254717s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:26:47 UTC) orbiter warning: context deadline exceeded Duration: 12.925277111s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [XX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....X.X] duration: 5.001249829s trace: https://jaeger.dev.storj.io/trace/7b7193cb1196312d FAIL (2026-01-14 19:26:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001268124s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.673680566s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:26:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.673715843s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....X.] duration: 3.432838852s trace: https://jaeger.dev.storj.io/trace/6c0f9c65c944c1f FAIL (2026-01-14 19:25:59 UTC) orbiter warning: context deadline exceeded Duration: 3.432862077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.420643071s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:25:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.42066349s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.879834757s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:25:37 UTC) orbiter warning: context deadline exceeded Duration: 5.879860045s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.831901872s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:25:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.8319316s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 23.315302138s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:25:23 UTC) orbiter warning: context deadline exceeded Duration: 23.315325191s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....X] duration: 5.001452845s trace: https://jaeger.dev.storj.io/trace/6c0f9c65c944c1f FAIL (2026-01-14 19:25:02 UTC) orbiter warning: context deadline exceeded Duration: 5.001467042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 15.090688058s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:24:38 UTC) orbiter warning: context deadline exceeded Duration: 15.090712714s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 8.106449354s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:24:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 8.106462629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.759309784s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:24:11 UTC) orbiter warning: context deadline exceeded Duration: 5.759333338s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [X.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X....] duration: 4.604437672s trace: https://jaeger.dev.storj.io/trace/5329f6fdfc11c57e FAIL (2026-01-14 19:24:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.604468641s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.917989931s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:23:52 UTC) orbiter warning: context deadline exceeded Duration: 6.918014257s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 7.093507379s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:23:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.093533449s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 27.537122521s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:23:31 UTC) orbiter warning: context deadline exceeded Duration: 27.537140846s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 8.722175337s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:23:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 8.722202649s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X...] duration: 3.63918941s trace: https://jaeger.dev.storj.io/trace/5329f6fdfc11c57e FAIL (2026-01-14 19:22:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.639213235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X..] duration: 3.90224281s trace: https://jaeger.dev.storj.io/trace/5329f6fdfc11c57e FAIL (2026-01-14 19:22:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.902270343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.322573574s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:22:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.322599795s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.038120865s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:22:13 UTC) orbiter warning: context deadline exceeded Duration: 6.038151885s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X.] duration: 4.557008111s trace: https://jaeger.dev.storj.io/trace/5329f6fdfc11c57e FAIL (2026-01-14 19:22:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.557032648s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 12.909375067s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:21:55 UTC) orbiter warning: context deadline exceeded Duration: 12.90939797s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..X] duration: 5.000883101s trace: https://jaeger.dev.storj.io/trace/5329f6fdfc11c57e FAIL (2026-01-14 19:21:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000899824s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 8.711661744s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:21:20 UTC) orbiter warning: context deadline exceeded Duration: 8.711687803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.351483533s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:21:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.351546503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.045831837s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:20:27 UTC) orbiter warning: context deadline exceeded Duration: 6.045857596s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.301988831s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:20:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.302019819s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [X.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X..] duration: 3.604531829s trace: https://jaeger.dev.storj.io/trace/e9cb508945dd9d3 FAIL (2026-01-14 19:20:11 UTC) orbiter warning: context deadline exceeded Duration: 3.604568428s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.540504279s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:19:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.540530539s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.979789345s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:19:35 UTC) orbiter warning: context deadline exceeded Duration: 8.979803402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.479376678s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:19:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.479404711s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.246858683s trace: https://jaeger.dev.storj.io/trace/54245678f503fe07 FAIL (2026-01-14 19:19:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 23.100507356 (963h49m22.971590391s) [upload=843773]: Got next segment. Inline: false Jan 14 19:18:23.100543044 (963h49m22.971626079s) [upload=843773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 14 19:18:23.100547001 (963h49m22.971630036s) [upload=843773]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 14 19:18:23.338845038 (963h49m23.209928073s) [seg_pos={0 0}, upload=843773]: Begin upload segment... Jan 14 19:18:23.338893670 (963h49m23.209976715s) [seg_pos={0 0}, upload=843773]: Done begin upload segment. Jan 14 19:18:23.338918177 (963h49m23.210001212s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339012858 (963h49m23.210095893s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339607520 (963h49m23.210690555s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.339904395 (963h49m23.21098743s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340197293 (963h49m23.211280328s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340525768 (963h49m23.211608803s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.340878710 (963h49m23.211961745s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341235008 (963h49m23.212318054s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.341770038 (963h49m23.212853073s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342280649 (963h49m23.213363685s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.342646917 (963h49m23.213729952s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343584462 (963h49m23.214667507s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.343737073 (963h49m23.214820108s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.344914585 (963h49m23.21599762s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346084553 (963h49m23.217167588s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.346201214 (963h49m23.217284249s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347239962 (963h49m23.218322997s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:23.347432099 (963h49m23.218515134s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Uploading piece... Jan 14 19:18:24.800207797 (963h49m24.671290832s) [upload=843773]: Next returned nil segment Jan 14 19:18:24.800241421 (963h49m24.671324486s) [upload=843773]: Waiting for error group managing segments... Jan 14 19:18:25.204457858 (963h49m25.075540913s) [noise=true , address=77.90.31.15:4705 , piece_id=RRQVS3DJ6NWU6NMD3STPDVRRZWDIRUQVK64NWLG2JU6JNNI4JXZA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.205544167 (963h49m25.076627202s) [noise=true , address=145.79.227.20:4450 , piece_id=AUD2PXONJPVKYBDU774SK5SOASTT4JBYCXMLJVCPKGYRONVG4WWQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.231510509 (963h49m25.102593554s) [noise=true , address=145.79.227.28:4755 , piece_id=T2QWW2U6JM54KN25OK6E2E7KTSW6ZXW5OASW4A2X2PXZNL2MVKMQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.238292923 (963h49m25.109375958s) [noise=true , address=145.79.227.36:4544 , piece_id=EJ7GPN4ZZG6JYFQJQVTCXFTNDYY3TAF3GH6QDAAMNIYQM4VP6GYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316338057 (963h49m25.187421092s) [noise=true , address=145.79.227.31:4814 , piece_id=ZAQVCYZZ4ZMJ7YU6SG7LDHXIMFXEDOMRQM3T4IKDQNBMW3UOA77A, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.316612379 (963h49m25.187695414s) [noise=true , address=145.79.227.38:4011 , piece_id=Y3F4N6FAZAEORIXQAYL6TOWB7ZH6EZBMO5YUCNMCLV5PEOAAT6RQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.318905755 (963h49m25.18998879s) [noise=true , address=45.91.116.4:4970 , piece_id=ZX57ZLKANZFAR7LWDIQTK2Q7FZFDFNOINA2AJ6UNYNUHKQJGAFRQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.319384868 (963h49m25.190467903s) [noise=true , address=45.91.116.6:4992 , piece_id=HTMLUCO43WZPPP67ZPAZKUEUURCGVFI5CECBIMEQQMKHGUVYMLBA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.359815310 (963h49m25.230898355s) [noise=true , address=145.79.227.30:4789 , piece_id=QE3ZE2MHEUKTIXW7CD3L6YLW3T5PQDGMX53IIWJBLCJNFXX2RFYA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:25.858715592 (963h49m25.729798628s) [noise=true , address=77.90.31.14:4689 , piece_id=E4A3XX5BXIZO3X2ROBRYBGGNQYFSWPLDI7GBP7CKBSQWL6ESP7QA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.169388345 (963h49m26.040471381s) [noise=true , address=77.90.31.10:4526 , piece_id=26HG627C76EOSMDY6QBIQL3IYW3XLWR5UMAWPAV7JVDJ66RMPARQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.784621894 (963h49m26.655704929s) [noise=true , address=77.90.31.19:4853 , piece_id=NUOAPQDXDXDU4YPKR2OHZCIIUDFGAV6JWN7QYZWGTEYGNHOBUJHA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:18:26.791209137 (963h49m26.662292172s) [noise=true , address=77.90.31.21:4903 , piece_id=GRHTMTNOWDONER6SYBZOA4LMB4SOVDR75QZZXGAKA4GQECAWKNEQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: <nil> Jan 14 19:19:22.436924793 (963h50m22.308007828s) [noise=true , address=5.231.73.4:4021 , piece_id=232PJMA5UK4OUQ6YFELY7VN3I6LBKHUW4G6KAD3TVITHWUEB5REA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1dXfFE3nY9ZSEixFePGxdptsrybf38kYjYgV61MzV8kHtc12w, address:5.231.73.4:4021): protocol: rpc: dial tcp 5.231.73.4:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437128741 (963h50m22.308211776s) [noise=true , address=5.231.73.2:4024 , piece_id=CXPXAWHQN4IFUR7VQ56YQ3Y4LGXA6SV73W2LE67A6KEZETIZISFA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:19eaYeTz1sE3KG63t5zvcd1n4wUDSx97a36UFkPMaNdGzDr92y, address:5.231.73.2:4024): protocol: rpc: dial tcp 5.231.73.2:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437138670 (963h50m22.308221705s) [noise=true , address=5.231.73.4:4015 , piece_id=QI6M4T7X3WDNZYVHCAUNRPZKNYC7QYSIJ2E7JU6BCLS5FTICGRNA, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:1k6f1Y37Ttp69PvjLUqXTZbgMTNJYPakDQDDVg2iS2xSTNoCxr, address:5.231.73.4:4015): protocol: read tcp 188.166.225.170:33908->5.231.73.4:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437442188 (963h50m22.308525223s) [noise=true , address=5.231.73.2:4007 , piece_id=KZU2UC7BKTKG66FAZ5WQG65GXPE5PY64LOFA3AIOKIFWCKT77HNQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12LpEnNKHi8R8uGooFw5enWsmfgqcvmjiTxJSzCeAMFB7c4Ythq, address:5.231.73.2:4007): protocol: read tcp 188.166.225.170:37618->5.231.73.2:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437493085 (963h50m22.30857612s) [noise=true , address=5.231.73.5:4019 , piece_id=VZYNV6FCKOSP7EUYCQ7C7AXB6GIM2RMMT5TPHYGL3EOJAYEDUJPQ, seg_pos={0 0}, upload=843773]: Done uploading piece. err: ecclient: upload failed (node:12uYwKKeXmjaZg9GnVTKPvb2h32vj3DHBpJu6vLWruVDZ5TAW8K, address:5.231.73.5:4019): protocol: rpc: dial tcp 5.231.73.5:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 14 19:19:22.437516119 (963h50m22.308599154s) [seg_pos={0 0}, upload=843773]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jan 14 19:19:22.681883964 (963h50m22.552966999s) [upload=843773]: Done uploading segments... Jan 14 19:19:22.681917808 (963h50m22.553000853s) [upload=843773]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.246881177s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 30 out of 60 is failed (threshold=6) [XX.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X.] duration: 3.514878389s trace: https://jaeger.dev.storj.io/trace/e9cb508945dd9d3 FAIL (2026-01-14 19:19:01 UTC) orbiter warning: context deadline exceeded Duration: 3.514906764s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.185484892s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:18:46 UTC) orbiter warning: context deadline exceeded Duration: 6.185514117s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.700587137s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:18:03 UTC) orbiter warning: context deadline exceeded Duration: 5.700616915s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [XXX.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....X] duration: 5.001163715s trace: https://jaeger.dev.storj.io/trace/e9cb508945dd9d3 FAIL (2026-01-14 19:18:01 UTC) orbiter warning: context deadline exceeded Duration: 5.00118704s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [XXXX.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X....] duration: 3.568968403s trace: https://jaeger.dev.storj.io/trace/3caef5c24001e682 FAIL (2026-01-14 19:17:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.568996486s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.194911307s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:17:22 UTC) orbiter warning: context deadline exceeded Duration: 6.19492327s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.918071774s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:17:05 UTC) orbiter warning: context deadline exceeded Duration: 5.918097564s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [XXXXX.X.XX.XXX..X.XXXXXX.X.X.XXXX...XX.XX....X.X......X.X...] duration: 3.95769668s trace: https://jaeger.dev.storj.io/trace/3caef5c24001e682 FAIL (2026-01-14 19:16:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.957788886s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 13.01578974s trace: https://jaeger.dev.storj.io/trace/12e802d7d7c733cb FAIL (2026-01-14 19:16:30 UTC) orbiter warning: context deadline exceeded Duration: 13.015816441s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)