Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 331621 reports (Page 6322 of 6633)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.................X..X.....................X...X.............] duration: 49.300433044s FAIL (2025-07-09 20:12:41 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 49.300463122s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.613862297s FAIL (2025-07-09 20:12:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.613886393s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..................X..X.....................X...X............] duration: 24.971066042s FAIL (2025-07-09 20:11:31 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 24.971096501s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.......................X...................................] duration: 6.139496514s FAIL (2025-07-09 20:11:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.13952552s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [............................X.........X..X...X........XX....] duration: 31.249343541s FAIL (2025-07-09 20:11:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 31.24935355s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...................X..X.....................X...X...........] duration: 17.24921977s FAIL (2025-07-09 20:11:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 17.249245789s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.............................X.........X..X...X........XX...] duration: 55.401683293s FAIL (2025-07-09 20:10:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 55.401711195s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.......................X..................................] duration: 6.452117786s FAIL (2025-07-09 20:10:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.452140178s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....................X..X.....................X...X..........] duration: 17.927221864s FAIL (2025-07-09 20:10:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 17.927237235s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....................X..X.....................X...X.........] duration: 50.796982672s FAIL (2025-07-09 20:10:18 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 50.796999734s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.......................X.................................] duration: 6.153098064s FAIL (2025-07-09 20:10:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.153120776s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..............................X.........X..X...X........XX..] duration: 32.657253276s FAIL (2025-07-09 20:09:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 32.657305044s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.......................X................................] duration: 5.971191425s FAIL (2025-07-09 20:09:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.971218054s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......................X..X.....................X...X........] duration: 45.399393456s FAIL (2025-07-09 20:09:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 45.399420498s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...............................X.........X..X...X........XX.] duration: 37.594915229s FAIL (2025-07-09 20:08:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 37.594936101s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.......................X...............................] duration: 6.496887517s FAIL (2025-07-09 20:08:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.49691014s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......................X..X.....................X...X.......] duration: 36.020793789s FAIL (2025-07-09 20:08:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 36.020817565s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [................................X.........X..X...X........XX] duration: 1m0.000760164s FAIL (2025-07-09 20:07:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000782046s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.......................X..............................] duration: 5.962505866s FAIL (2025-07-09 20:07:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.962527026s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................X..X.....................X...X......] duration: 13.005477257s FAIL (2025-07-09 20:07:09 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 13.005510038s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.......................X.............................] duration: 6.349121391s FAIL (2025-07-09 20:07:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.349143111s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.................................X.........X..X...X........X] duration: 1m0.001508476s FAIL (2025-07-09 20:06:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001532192s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.........................X..X.....................X...X.....] duration: 10.384929029s FAIL (2025-07-09 20:06:45 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 10.384954997s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..........................X..X.....................X...X....] duration: 44.821953614s FAIL (2025-07-09 20:06:13 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 44.821979012s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.......................X............................] duration: 6.06641572s FAIL (2025-07-09 20:06:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.066438072s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..................................X.........X..X...X........] duration: 47.442832781s FAIL (2025-07-09 20:05:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 47.442855386s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...........................X..X.....................X...X...] duration: 29.960340332s FAIL (2025-07-09 20:05:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 29.960367665s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.......................X...........................] duration: 6.245009717s FAIL (2025-07-09 20:05:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.245033733s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...................................X.........X..X...X.......] duration: 40.985134954s FAIL (2025-07-09 20:04:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 40.985159521s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X........X.......................X..........................] duration: 6.223252321s FAIL (2025-07-09 20:04:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.22372273s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [............................X..X.....................X...X..] duration: 19.982213273s FAIL (2025-07-09 20:04:09 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 19.982239293s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....................................X.........X..X...X......] duration: 19.080896996s FAIL (2025-07-09 20:03:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 19.080908458s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.............................X..X.....................X...X.] duration: 14.93061122s FAIL (2025-07-09 20:03:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 14.930626469s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X........X.......................X.........................] duration: 6.247300267s FAIL (2025-07-09 20:03:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.247319523s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X.........X..X...X.....] duration: 35.934388695s FAIL (2025-07-09 20:03:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 35.934404385s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..............................X..X.....................X...X] duration: 1m0.001959452s FAIL (2025-07-09 20:03:22 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== . Jul 9 20:02:23.780217498 (394h51m58.550858883s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780676316 (394h51m58.551317641s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.780936945 (394h51m58.55157827s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781177245 (394h51m58.55181858s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781413888 (394h51m58.552055213s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781644208 (394h51m58.552285543s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.781895629 (394h51m58.552536954s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782135739 (394h51m58.552777074s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782318628 (394h51m58.552959953s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782512379 (394h51m58.553153714s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.782777476 (394h51m58.553418801s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783024369 (394h51m58.553665704s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783212469 (394h51m58.553853804s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783488777 (394h51m58.554130102s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783681907 (394h51m58.554323232s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.783865978 (394h51m58.554507313s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784048738 (394h51m58.554690073s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:23.784240856 (394h51m58.554882191s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Uploading piece... Jul 9 20:02:24.899970585 (394h51m59.67061192s) [upload=315928]: Next returned nil segment Jul 9 20:02:24.899992798 (394h51m59.670634133s) [upload=315928]: Waiting for error group managing segments... Jul 9 20:02:25.204965397 (394h51m59.975606732s) [noise=true , address=79.127.205.235:4669 , piece_id=YBAEIQ4D342ZRZNZ7CN3FP5XXZEP4JRTSITTIETCFQCG52TMXZLA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.767637823 (394h52m0.538279158s) [noise=true , address=79.127.219.38:4008 , piece_id=5QBBOWQMUYCAT4KY6DSPIFS2J4D7FOV24RGMT3FNJYOTL6X764QQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.775539020 (394h52m0.546180355s) [noise=true , address=77.90.31.15:4710 , piece_id=LD6INE3ECX3HSS7CX25RWQYXE2PUQSUWZVQ3VVMINCKY64JQRV4Q, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.784443085 (394h52m0.55508442s) [noise=true , address=77.90.31.10:4531 , piece_id=3IHUYY7Q6CJH23BFCDYJYWEPVOWL7VXCO5W727VHMTL2RTJYA4FA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.786360713 (394h52m0.557002048s) [noise=true , address=77.90.31.18:4821 , piece_id=JIUP3CSGG23DG6XVVQOSFXPXM2PFZTKEJTZ6O6IKE4MHZ2XAUE2A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.790650711 (394h52m0.561292046s) [noise=true , address=77.90.31.14:4682 , piece_id=EXXWHHHRTELAI2WW6472EOUDCWAXTA5H5MLTUF5B5FJOU5NLMEOA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.795596423 (394h52m0.566237758s) [noise=true , address=77.90.31.16:4748 , piece_id=RARHNZP44WW3V4LB5TGMZUCGVWODY5LBJUTYU6UGPPJINALZA5IQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.800154773 (394h52m0.570796108s) [noise=true , address=77.90.31.13:4646 , piece_id=XURX6FTRDRDPEVUDINT3UCBCZYJB2C6K6BHS3LNKV4LYM22AAFWA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:25.809266435 (394h52m0.57990776s) [noise=true , address=77.90.31.16:4768 , piece_id=EADGMMZXL27TH2ZPFSR7QFWWEVWQEVFITN3TAU2TM2OVN64IBJSA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:33.323814508 (394h52m8.094455843s) [noise=true , address=79.127.201.215:4348 , piece_id=PYWYROAV33776ZW4O3U3SIE4VNI4IZUL6RUZOEP3QR4NZNLFG46A, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:02:54.619833778 (394h52m29.390475153s) [noise=true , address=104.36.82.2:4746 , piece_id=ORV3PXJ6LHH4J5PTYQ7UMGU36H2PWKKJMTUHZSL7CMCWKDMBBMNQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:03.031749677 (394h52m37.802391012s) [noise=true , address=104.36.82.5:4863 , piece_id=4EOE6YQGQVXDF26VILTGTHLB422UDGVLEXBMG3344WIAMR6WBWGQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:08.267656661 (394h52m43.038297996s) [noise=true , address=104.36.82.51:4064 , piece_id=WQSKOGGEF7KROPVU62HVACPS74QYDHYZQUZIXXZ6Q6APKCTNBCZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696211223 (394h52m52.466852548s) [noise=true , address=104.36.82.39:4738 , piece_id=7QVQHRYSHG2XKLZ5AJROBO7CI5COBRPN7B6JFYC2QMNUUUR35QYA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: <nil> Jul 9 20:03:17.696238234 (394h52m52.466879569s) [seg_pos={0 0}, upload=315928]: Segment reached optimal threshold of 14 pieces. Jul 9 20:03:17.696433668 (394h52m52.467075003s) [noise=true , address=104.36.82.19:4426 , piece_id=NYUABLYZULXKVJKDVAJD35VBQKUEFOAZO3SJ73GYBEWI5AFH5LHA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f73dFC5Avj5HvkthE72J8wsV8b6xWAH9gVLqGEThS2b9rTjPk): protocol: expected piece hash; context canceled; EOF Jul 9 20:03:17.696504703 (394h52m52.467146038s) [noise=true , address=104.36.82.36:4617 , piece_id=QNSC3HECIXLZ62BBH4LXBVN3UR7DZWOZ64VFP7AIFO2KH4DDT4ZQ, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw): protocol: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45496->104.36.82.36:4617: use of closed network connection Jul 9 20:03:17.696540202 (394h52m52.467181537s) [noise=true , address=104.36.82.38:4707 , piece_id=IYK43BYSQUNEQ6BAUO245LMNAOBGGV7TGB76I4FKC6MABPNMBUDA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T): protocol: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37852->104.36.82.38:4707: use of closed network connection Jul 9 20:03:17.696549359 (394h52m52.467190694s) [noise=true , address=104.36.82.23:4445 , piece_id=J6VPRBOQSJIOEOM2LZJBQJJYCXXSBTIIVDZGYIBDB7UNRT5CA7QA, seg_pos={0 0}, upload=315928]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo): protocol: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36278->104.36.82.23:4445: use of closed network connection Jul 9 20:03:17.696563035 (394h52m52.46720437s) [seg_pos={0 0}, upload=315928]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 20:03:17.696661103 (394h52m52.467302428s) [upload=315928]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:03:17.696665161 (394h52m52.467306496s) [upload=315928]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:03:18.153388631 (394h52m52.924029966s) [upload=315928]: Done uploading segments... Jul 9 20:03:18.153408019 (394h52m52.924049354s) [upload=315928]: Upload finished. err: <nil> Duration: 1m0.001980052s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......................................X.........X..X...X....] duration: 50.90247921s FAIL (2025-07-09 20:02:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 50.90250462s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X........X.......................X........................] duration: 6.290626239s FAIL (2025-07-09 20:02:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.290650986s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X........X.......................X.......................] duration: 6.172879658s FAIL (2025-07-09 20:02:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.17290166s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................X..X.....................X...] duration: 10.623529762s FAIL (2025-07-09 20:02:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== [upload=315898]: Got next segment. Inline: false Jul 9 20:00:01.267693251 (394h49m36.038334586s) [upload=315898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:00:01.267700135 (394h49m36.03834147s) [upload=315898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:00:01.715201952 (394h49m36.485843307s) [seg_pos={0 0}, upload=315898]: Begin upload segment... Jul 9 20:00:01.715296183 (394h49m36.485937528s) [seg_pos={0 0}, upload=315898]: Done begin upload segment. Jul 9 20:00:01.715332582 (394h49m36.485973937s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715353733 (394h49m36.485995078s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715413648 (394h49m36.486054993s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715839523 (394h49m36.486480868s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715854491 (394h49m36.486495836s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716252824 (394h49m36.486894169s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716274825 (394h49m36.48691617s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716770403 (394h49m36.487411768s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716807425 (394h49m36.48744877s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717175559 (394h49m36.487816904s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717261513 (394h49m36.487902858s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717562689 (394h49m36.488204034s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717649675 (394h49m36.488291041s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718081622 (394h49m36.488722967s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718456659 (394h49m36.489098004s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.719830938 (394h49m36.490472294s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.720421469 (394h49m36.491062864s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.721072935 (394h49m36.49171437s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:02.846673368 (394h49m37.617314713s) [upload=315898]: Next returned nil segment Jul 9 20:00:02.846727541 (394h49m37.617368896s) [upload=315898]: Waiting for error group managing segments... Jul 9 20:00:03.434946994 (394h49m38.205588339s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.629357310 (394h49m38.399998645s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.710145991 (394h49m38.480787336s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.724809153 (394h49m38.495450499s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.726766118 (394h49m38.497407463s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.736459613 (394h49m38.507100948s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.737328855 (394h49m38.5079702s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.744352673 (394h49m38.514994018s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.747871785 (394h49m38.518513131s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:07.148967050 (394h49m41.919608385s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:09.061157378 (394h49m43.831798713s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:13.841876001 (394h49m48.612517336s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:45.484870134 (394h50m20.255511469s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:01:00.622075620 (394h50m35.392716945s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y, address:104.36.82.36:4631): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622243040 (394h50m35.392884375s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:15y4L1tWUv5vFniK5ugNHNYa22q7a7hC5Uf9pxkN4AuPBodcgY, address:104.36.82.20:4462): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622283868 (394h50m35.392925203s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:104.36.82.44:4873): protocol: expected piece hash; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; EOF Jul 9 20:01:00.622537793 (394h50m35.393179128s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:104.36.82.17:4332): protocol: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection Jul 9 20:01:00.622584353 (394h50m35.393225688s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:104.36.82.34:4546): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622614701 (394h50m35.393256036s) [seg_pos={0 0}, upload=315898]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:01:00.859090035 (394h50m35.62973137s) [upload=315898]: Done uploading segments... Jul 9 20:01:00.859115674 (394h50m35.629757009s) [upload=315898]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.62355961s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................................X..X.....................X..] duration: 19.455972537s FAIL (2025-07-09 20:01:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== [upload=315898]: Got next segment. Inline: false Jul 9 20:00:01.267693251 (394h49m36.038334586s) [upload=315898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:00:01.267700135 (394h49m36.03834147s) [upload=315898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:00:01.715201952 (394h49m36.485843307s) [seg_pos={0 0}, upload=315898]: Begin upload segment... Jul 9 20:00:01.715296183 (394h49m36.485937528s) [seg_pos={0 0}, upload=315898]: Done begin upload segment. Jul 9 20:00:01.715332582 (394h49m36.485973937s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715353733 (394h49m36.485995078s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715413648 (394h49m36.486054993s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715839523 (394h49m36.486480868s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715854491 (394h49m36.486495836s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716252824 (394h49m36.486894169s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716274825 (394h49m36.48691617s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716770403 (394h49m36.487411768s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716807425 (394h49m36.48744877s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717175559 (394h49m36.487816904s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717261513 (394h49m36.487902858s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717562689 (394h49m36.488204034s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717649675 (394h49m36.488291041s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718081622 (394h49m36.488722967s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718456659 (394h49m36.489098004s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.719830938 (394h49m36.490472294s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.720421469 (394h49m36.491062864s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.721072935 (394h49m36.49171437s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:02.846673368 (394h49m37.617314713s) [upload=315898]: Next returned nil segment Jul 9 20:00:02.846727541 (394h49m37.617368896s) [upload=315898]: Waiting for error group managing segments... Jul 9 20:00:03.434946994 (394h49m38.205588339s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.629357310 (394h49m38.399998645s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.710145991 (394h49m38.480787336s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.724809153 (394h49m38.495450499s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.726766118 (394h49m38.497407463s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.736459613 (394h49m38.507100948s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.737328855 (394h49m38.5079702s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.744352673 (394h49m38.514994018s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.747871785 (394h49m38.518513131s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:07.148967050 (394h49m41.919608385s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:09.061157378 (394h49m43.831798713s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:13.841876001 (394h49m48.612517336s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:45.484870134 (394h50m20.255511469s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:01:00.622075620 (394h50m35.392716945s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y, address:104.36.82.36:4631): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622243040 (394h50m35.392884375s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:15y4L1tWUv5vFniK5ugNHNYa22q7a7hC5Uf9pxkN4AuPBodcgY, address:104.36.82.20:4462): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622283868 (394h50m35.392925203s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:104.36.82.44:4873): protocol: expected piece hash; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; EOF Jul 9 20:01:00.622537793 (394h50m35.393179128s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:104.36.82.17:4332): protocol: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection Jul 9 20:01:00.622584353 (394h50m35.393225688s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:104.36.82.34:4546): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622614701 (394h50m35.393256036s) [seg_pos={0 0}, upload=315898]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:01:00.859090035 (394h50m35.62973137s) [upload=315898]: Done uploading segments... Jul 9 20:01:00.859115674 (394h50m35.629757009s) [upload=315898]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.455990962s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.........X..X...X...] duration: 23.322246158s FAIL (2025-07-09 20:01:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.322285594s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X........X.......................X......................] duration: 6.762320478s FAIL (2025-07-09 20:01:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.762344594s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.................................X..X.....................X.] duration: 28.537369929s FAIL (2025-07-09 20:01:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== [upload=315898]: Got next segment. Inline: false Jul 9 20:00:01.267693251 (394h49m36.038334586s) [upload=315898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:00:01.267700135 (394h49m36.03834147s) [upload=315898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:00:01.715201952 (394h49m36.485843307s) [seg_pos={0 0}, upload=315898]: Begin upload segment... Jul 9 20:00:01.715296183 (394h49m36.485937528s) [seg_pos={0 0}, upload=315898]: Done begin upload segment. Jul 9 20:00:01.715332582 (394h49m36.485973937s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715353733 (394h49m36.485995078s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715413648 (394h49m36.486054993s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715839523 (394h49m36.486480868s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715854491 (394h49m36.486495836s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716252824 (394h49m36.486894169s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716274825 (394h49m36.48691617s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716770403 (394h49m36.487411768s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716807425 (394h49m36.48744877s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717175559 (394h49m36.487816904s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717261513 (394h49m36.487902858s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717562689 (394h49m36.488204034s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717649675 (394h49m36.488291041s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718081622 (394h49m36.488722967s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718456659 (394h49m36.489098004s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.719830938 (394h49m36.490472294s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.720421469 (394h49m36.491062864s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.721072935 (394h49m36.49171437s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:02.846673368 (394h49m37.617314713s) [upload=315898]: Next returned nil segment Jul 9 20:00:02.846727541 (394h49m37.617368896s) [upload=315898]: Waiting for error group managing segments... Jul 9 20:00:03.434946994 (394h49m38.205588339s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.629357310 (394h49m38.399998645s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.710145991 (394h49m38.480787336s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.724809153 (394h49m38.495450499s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.726766118 (394h49m38.497407463s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.736459613 (394h49m38.507100948s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.737328855 (394h49m38.5079702s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.744352673 (394h49m38.514994018s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.747871785 (394h49m38.518513131s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:07.148967050 (394h49m41.919608385s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:09.061157378 (394h49m43.831798713s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:13.841876001 (394h49m48.612517336s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:45.484870134 (394h50m20.255511469s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:01:00.622075620 (394h50m35.392716945s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y, address:104.36.82.36:4631): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622243040 (394h50m35.392884375s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:15y4L1tWUv5vFniK5ugNHNYa22q7a7hC5Uf9pxkN4AuPBodcgY, address:104.36.82.20:4462): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622283868 (394h50m35.392925203s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:104.36.82.44:4873): protocol: expected piece hash; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; EOF Jul 9 20:01:00.622537793 (394h50m35.393179128s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:104.36.82.17:4332): protocol: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection Jul 9 20:01:00.622584353 (394h50m35.393225688s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:104.36.82.34:4546): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622614701 (394h50m35.393256036s) [seg_pos={0 0}, upload=315898]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:01:00.859090035 (394h50m35.62973137s) [upload=315898]: Done uploading segments... Jul 9 20:01:00.859115674 (394h50m35.629757009s) [upload=315898]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 28.537389508s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................................X.........X..X...X..] duration: 53.662904797s FAIL (2025-07-09 20:01:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 53.66292736s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.83391143s FAIL (2025-07-09 20:01:18 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 4.833935147s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................................X..X.....................X] duration: 1m0.238222308s FAIL (2025-07-09 20:01:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== [upload=315898]: Got next segment. Inline: false Jul 9 20:00:01.267693251 (394h49m36.038334586s) [upload=315898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:00:01.267700135 (394h49m36.03834147s) [upload=315898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:00:01.715201952 (394h49m36.485843307s) [seg_pos={0 0}, upload=315898]: Begin upload segment... Jul 9 20:00:01.715296183 (394h49m36.485937528s) [seg_pos={0 0}, upload=315898]: Done begin upload segment. Jul 9 20:00:01.715332582 (394h49m36.485973937s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715353733 (394h49m36.485995078s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715413648 (394h49m36.486054993s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715839523 (394h49m36.486480868s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.715854491 (394h49m36.486495836s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716252824 (394h49m36.486894169s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716274825 (394h49m36.48691617s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716770403 (394h49m36.487411768s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.716807425 (394h49m36.48744877s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717175559 (394h49m36.487816904s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717261513 (394h49m36.487902858s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717562689 (394h49m36.488204034s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.717649675 (394h49m36.488291041s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718081622 (394h49m36.488722967s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.718456659 (394h49m36.489098004s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.719830938 (394h49m36.490472294s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.720421469 (394h49m36.491062864s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:01.721072935 (394h49m36.49171437s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Uploading piece... Jul 9 20:00:02.846673368 (394h49m37.617314713s) [upload=315898]: Next returned nil segment Jul 9 20:00:02.846727541 (394h49m37.617368896s) [upload=315898]: Waiting for error group managing segments... Jul 9 20:00:03.434946994 (394h49m38.205588339s) [noise=true , address=79.127.220.101:4062 , piece_id=SC2KCK7S4327ZJB3ILAUIL5Q2XWI2J4GBHFPQX7SRLOXLHPOE5AA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.629357310 (394h49m38.399998645s) [noise=true , address=79.127.201.215:4342 , piece_id=ITAJMMOKLXOC3F53AYANHCGR3EVVYJEA5KBFJRACFDZ6F3LSPEZA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.710145991 (394h49m38.480787336s) [noise=true , address=77.90.31.12:4611 , piece_id=3KC25O7DV2CKTIRPMORWUQQ4VERAOXR7XBWKW3IODAB5XOGMEQRA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.724809153 (394h49m38.495450499s) [noise=true , address=77.90.31.21:4902 , piece_id=RXRCF4IUXTPNSCBNRDSGGWBTZ4U4PJBGFQ73WZGTWURMWYHPTTMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.726766118 (394h49m38.497407463s) [noise=true , address=77.90.31.16:4739 , piece_id=VFIPXNATKRTP26LZG764GSCDXHGLOYXJ5NEDD5Q5PG6KJ2OJG2TQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.736459613 (394h49m38.507100948s) [noise=true , address=77.90.31.10:4547 , piece_id=VYBGPSHLJRHQO6I4YVBVUKLEOCOHKRDEJX73E42KMR34HFRAXRVQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.737328855 (394h49m38.5079702s) [noise=true , address=77.90.31.17:4769 , piece_id=GU7GSQUWVJU3OJRQO5L6RBIP7HEXVFG7XVOGXTE5W3COGQXIXFDA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.744352673 (394h49m38.514994018s) [noise=true , address=77.90.31.19:4857 , piece_id=FB6WLVLF5HNUD6ZIUIBAT76MDR25SWC6DGGSOYNMDVCDX4R52S7Q, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:03.747871785 (394h49m38.518513131s) [noise=true , address=77.90.31.15:4724 , piece_id=NC4ITOJ2TPWCDEQRG2UNFDATYDHSY6WEOAI6O7WSYT2DPKC75KKQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:07.148967050 (394h49m41.919608385s) [noise=true , address=79.127.201.215:4355 , piece_id=6WYNZ5YDULZ4IHUIQLZIQES5GMCXSXXTWIFK6RY4PSIGZX55IQMQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:09.061157378 (394h49m43.831798713s) [noise=true , address=79.127.219.47:4047 , piece_id=BSVCIZGYNCRVNPUP344AUGFZXYSIGJKMAU45ZYUWOFALNVDPA2PQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:13.841876001 (394h49m48.612517336s) [noise=true , address=79.127.219.38:4024 , piece_id=7QZOGDYRLNRPFYRHAU3FA52ZC4LUGHWMZSCGYOQ6P7KS5FOZ2IXA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:00:45.484870134 (394h50m20.255511469s) [noise=true , address=64.49.13.169:4706 , piece_id=UUWT3LNG4JE5JLSJKIEJVHC6H767QNRI4V4CE3WN6CT3HMTJ46MA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: <nil> Jul 9 20:01:00.622075620 (394h50m35.392716945s) [noise=true , address=104.36.82.36:4631 , piece_id=VUQ25JOQ3HWQRFSMHELSZSUSLQGZEL2JEFJWD2D3SZHUJGW4H5FQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12T6FqqMunZ46QoxhHVWsn8NinZGWv1Yo7rNHSiK6s9hhuQtS6Y, address:104.36.82.36:4631): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622243040 (394h50m35.392884375s) [noise=true , address=104.36.82.20:4462 , piece_id=2MWIVCTHD4EJHGU6PGLEUDBDIMPAAQMC2S6XDRTEQAP3NLTL2BLA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:15y4L1tWUv5vFniK5ugNHNYa22q7a7hC5Uf9pxkN4AuPBodcgY, address:104.36.82.20:4462): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622283868 (394h50m35.392925203s) [noise=true , address=104.36.82.44:4873 , piece_id=JAJWWIKUN64LFQEFX57MND77Y7K67OD37U2HFIZ5U5VYLBAZQWLQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:104.36.82.44:4873): protocol: expected piece hash; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; write tcp 188.166.225.170:37508->104.36.82.44:4873: use of closed network connection; EOF Jul 9 20:01:00.622537793 (394h50m35.393179128s) [noise=true , address=104.36.82.17:4332 , piece_id=NUZ2IFWCPLA3UF64YDPKP6HH5O4GK6FUYUYY3FVL7ZIVFKJZNUAQ, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:104.36.82.17:4332): protocol: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51046->104.36.82.17:4332: use of closed network connection Jul 9 20:01:00.622584353 (394h50m35.393225688s) [noise=true , address=104.36.82.34:4546 , piece_id=XROJVC7Q7VCSNA5TSARULFGS753MPQEBDFG6QW2ZX5ENPHKBL3XA, seg_pos={0 0}, upload=315898]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:104.36.82.34:4546): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:01:00.622614701 (394h50m35.393256036s) [seg_pos={0 0}, upload=315898]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:01:00.859090035 (394h50m35.62973137s) [upload=315898]: Done uploading segments... Jul 9 20:01:00.859115674 (394h50m35.629757009s) [upload=315898]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.238241123s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X........X.......................X.....................] duration: 5.916964496s FAIL (2025-07-09 20:00:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.91698721s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.887642123s FAIL (2025-07-09 20:00:09 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 4.887673923s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.........................................X.........X..X...X.] duration: 27.220397921s FAIL (2025-07-09 19:59:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 27.220416918s core uplink us-select-1 us1-placement-20

1
...
6320
6321
6322
6323
6324
...
6633