Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X..............................X.X....] duration: 5.865461768s FAIL (2025-07-16 19:07:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.865490463s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.491162104s FAIL (2025-07-16 19:06:37 UTC) orbiter warning: context deadline exceeded Duration: 6.491186631s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X..............................X.X...] duration: 5.741306311s FAIL (2025-07-16 19:06:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.741334446s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.025177067s FAIL (2025-07-16 19:06:10 UTC) orbiter warning: context deadline exceeded Duration: 6.025200941s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X..............................X.X..] duration: 5.926635122s FAIL (2025-07-16 19:06:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.926660289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X..............................X.X.] duration: 5.820487821s FAIL (2025-07-16 19:05:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.820504795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X..............................X.X] duration: 30.231753838s FAIL (2025-07-16 19:05:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.231773055s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.253642773s FAIL (2025-07-16 19:04:57 UTC) orbiter warning: context deadline exceeded Duration: 6.253666049s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.267509026s FAIL (2025-07-16 19:04:04 UTC) orbiter warning: context deadline exceeded Duration: 6.267532371s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X..............................X.] duration: 11.318622926s FAIL (2025-07-16 19:03:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== pb.BatchRequestItem_SegmentBegin Jul 16 19:03:11.884371888 (561h52m46.655013223s) [seg_pos={0 0}, upload=449305]: Begin upload segment... Jul 16 19:03:11.884416153 (561h52m46.655057488s) [seg_pos={0 0}, upload=449305]: Done begin upload segment. Jul 16 19:03:11.884439036 (561h52m46.655080371s) [noise=true , address=45.91.116.11:4117 , piece_id=OLL5H6H7LI7KLHXJBKYHTKYJJBBAMBTUHFLVNFF7IGCBTLCKMEWQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.884494913 (561h52m46.655136248s) [noise=true , address=45.91.116.6:4975 , piece_id=CCRBVJH3JJ2S4RWBACGHGPI3PLG3QXQRVRRD7PRRVWVXLBAD54NA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.884781572 (561h52m46.655422897s) [noise=true , address=77.90.31.18:4806 , piece_id=HIZ5UWUX2NKUTFPMEXBKQWVQJNMGUKXKW664E3KWQY3CGE2EGM7Q, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885019527 (561h52m46.655660862s) [noise=true , address=77.90.31.12:4592 , piece_id=JRH475SGAPR6VWSR5LOBWD2MENEEP4VI4KRWA4EYCX6WPPZRXBPA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885041519 (561h52m46.655682854s) [noise=true , address=45.91.116.4:4938 , piece_id=XQ5QX4Y4KCAHAS5CU3TLOCTYYUCNANB3Q5RZLFW6ZTT4NOCSG5MQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885316314 (561h52m46.655957649s) [noise=true , address=77.90.31.16:4734 , piece_id=CQNZGGK22L3ASWB7B7JZVQFSGIQXSPSW7QTVFF2APBCRQTYUKKJA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885390757 (561h52m46.656032092s) [noise=true , address=77.90.31.11:4553 , piece_id=FJJZYD55ZYEU7SMIY6YUJG7NETWWZSDAWQ7MKLCF7WZKAJD6PSDQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885562756 (561h52m46.656204081s) [noise=true , address=104.36.82.13:4187 , piece_id=D32HIFFGXLJGPNJ3MWITHYBHY6PJMPROYNY24WC2MLSKGYETEISQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885696472 (561h52m46.656337797s) [noise=true , address=104.36.82.5:4840 , piece_id=6NH4QMQKGIIPBIX2KWJVOBAO3KBD5NLBR3TU5WLTLQTT5B5LRA6A, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885827152 (561h52m46.656468477s) [noise=true , address=77.90.31.13:4627 , piece_id=HO7HTXMXQIL5VQYZ6QW4JT32YS3FYDDC4P6OQM4YPOZETW2PH6TQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886076038 (561h52m46.656717373s) [noise=true , address=104.36.82.15:4258 , piece_id=KQXN75BSJBDZ5ADQ6DEIHGXHABYH4XTI3BWNYT4GP2JKTNIG65AQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886276822 (561h52m46.656918157s) [noise=true , address=77.90.31.17:4770 , piece_id=E5KK5NPBX7R2DDZHBFXJXMUW7NVJH5EMXC5P6PV4GMJK3YTNV4FQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886462738 (561h52m46.657104073s) [noise=true , address=45.91.116.9:4261 , piece_id=KV547AY2I3O7NCOC5XD4YUTBSEUDXXHTVHL5MZYITA6MNRHBLLTA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886653002 (561h52m46.657294337s) [noise=true , address=77.90.31.18:4807 , piece_id=LCQZNCW4VIPB4AXGKUMHFFVWKAJMQOUA2E46AGRADDOO34X7SWLQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886840391 (561h52m46.657481726s) [noise=true , address=104.36.82.12:4151 , piece_id=PSP3UE26YG2YDQYNGQWL3AE2ZWGTUUBI4JOU6F2AJJBR67YYP5QA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.887087203 (561h52m46.657728538s) [noise=true , address=104.36.82.16:4295 , piece_id=BLQ5CYCBLNORM7SN2UWCVVJTY7BZQ3FDNFDCHA6YDNF3A6P62SDQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:13.650420754 (561h52m48.421062089s) [upload=449305]: Next returned nil segment Jul 16 19:03:13.650442185 (561h52m48.42108352s) [upload=449305]: Waiting for error group managing segments... Jul 16 19:03:13.957490344 (561h52m48.72813168s) [noise=true , address=104.36.82.13:4187 , piece_id=D32HIFFGXLJGPNJ3MWITHYBHY6PJMPROYNY24WC2MLSKGYETEISQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:13.998011145 (561h52m48.76865248s) [noise=true , address=104.36.82.16:4295 , piece_id=BLQ5CYCBLNORM7SN2UWCVVJTY7BZQ3FDNFDCHA6YDNF3A6P62SDQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.033735749 (561h52m48.804377084s) [noise=true , address=104.36.82.15:4258 , piece_id=KQXN75BSJBDZ5ADQ6DEIHGXHABYH4XTI3BWNYT4GP2JKTNIG65AQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.043837948 (561h52m48.814479283s) [noise=true , address=45.91.116.6:4975 , piece_id=CCRBVJH3JJ2S4RWBACGHGPI3PLG3QXQRVRRD7PRRVWVXLBAD54NA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.056883559 (561h52m48.827524894s) [noise=true , address=45.91.116.11:4117 , piece_id=OLL5H6H7LI7KLHXJBKYHTKYJJBBAMBTUHFLVNFF7IGCBTLCKMEWQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.098272381 (561h52m48.868913716s) [noise=true , address=104.36.82.5:4840 , piece_id=6NH4QMQKGIIPBIX2KWJVOBAO3KBD5NLBR3TU5WLTLQTT5B5LRA6A, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.165604781 (561h52m48.936246116s) [noise=true , address=45.91.116.4:4938 , piece_id=XQ5QX4Y4KCAHAS5CU3TLOCTYYUCNANB3Q5RZLFW6ZTT4NOCSG5MQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.183514829 (561h52m48.954156164s) [noise=true , address=104.36.82.12:4151 , piece_id=PSP3UE26YG2YDQYNGQWL3AE2ZWGTUUBI4JOU6F2AJJBR67YYP5QA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.215594298 (561h52m48.986235633s) [noise=true , address=45.91.116.9:4261 , piece_id=KV547AY2I3O7NCOC5XD4YUTBSEUDXXHTVHL5MZYITA6MNRHBLLTA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:40.996425523 (561h53m15.767066858s) [noise=true , address=77.90.31.11:4553 , piece_id=FJJZYD55ZYEU7SMIY6YUJG7NETWWZSDAWQ7MKLCF7WZKAJD6PSDQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:03:40.996509955 (561h53m15.76715129s) [noise=true , address=77.90.31.12:4592 , piece_id=JRH475SGAPR6VWSR5LOBWD2MENEEP4VI4KRWA4EYCX6WPPZRXBPA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:03:40.996649712 (561h53m15.767291047s) [noise=true , address=77.90.31.16:4734 , piece_id=CQNZGGK22L3ASWB7B7JZVQFSGIQXSPSW7QTVFF2APBCRQTYUKKJA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection Jul 16 19:03:40.996700199 (561h53m15.767341534s) [noise=true , address=77.90.31.18:4806 , piece_id=HIZ5UWUX2NKUTFPMEXBKQWVQJNMGUKXKW664E3KWQY3CGE2EGM7Q, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection Jul 16 19:03:40.996724826 (561h53m15.767366161s) [noise=true , address=77.90.31.17:4770 , piece_id=E5KK5NPBX7R2DDZHBFXJXMUW7NVJH5EMXC5P6PV4GMJK3YTNV4FQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection Jul 16 19:03:40.996768891 (561h53m15.767410216s) [noise=true , address=77.90.31.13:4627 , piece_id=HO7HTXMXQIL5VQYZ6QW4JT32YS3FYDDC4P6OQM4YPOZETW2PH6TQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection Jul 16 19:03:40.996809488 (561h53m15.767450893s) [noise=true , address=77.90.31.18:4807 , piece_id=LCQZNCW4VIPB4AXGKUMHFFVWKAJMQOUA2E46AGRADDOO34X7SWLQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection Jul 16 19:03:40.996845206 (561h53m15.767486591s) [seg_pos={0 0}, upload=449305]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 16 19:03:41.230111736 (561h53m16.000753071s) [upload=449305]: Done uploading segments... Jul 16 19:03:41.230136874 (561h53m16.000778209s) [upload=449305]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration:* 11.318691437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X..............................X] duration: 30.235242559s FAIL (2025-07-16 19:03:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== pb.BatchRequestItem_SegmentBegin Jul 16 19:03:11.884371888 (561h52m46.655013223s) [seg_pos={0 0}, upload=449305]: Begin upload segment... Jul 16 19:03:11.884416153 (561h52m46.655057488s) [seg_pos={0 0}, upload=449305]: Done begin upload segment. Jul 16 19:03:11.884439036 (561h52m46.655080371s) [noise=true , address=45.91.116.11:4117 , piece_id=OLL5H6H7LI7KLHXJBKYHTKYJJBBAMBTUHFLVNFF7IGCBTLCKMEWQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.884494913 (561h52m46.655136248s) [noise=true , address=45.91.116.6:4975 , piece_id=CCRBVJH3JJ2S4RWBACGHGPI3PLG3QXQRVRRD7PRRVWVXLBAD54NA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.884781572 (561h52m46.655422897s) [noise=true , address=77.90.31.18:4806 , piece_id=HIZ5UWUX2NKUTFPMEXBKQWVQJNMGUKXKW664E3KWQY3CGE2EGM7Q, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885019527 (561h52m46.655660862s) [noise=true , address=77.90.31.12:4592 , piece_id=JRH475SGAPR6VWSR5LOBWD2MENEEP4VI4KRWA4EYCX6WPPZRXBPA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885041519 (561h52m46.655682854s) [noise=true , address=45.91.116.4:4938 , piece_id=XQ5QX4Y4KCAHAS5CU3TLOCTYYUCNANB3Q5RZLFW6ZTT4NOCSG5MQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885316314 (561h52m46.655957649s) [noise=true , address=77.90.31.16:4734 , piece_id=CQNZGGK22L3ASWB7B7JZVQFSGIQXSPSW7QTVFF2APBCRQTYUKKJA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885390757 (561h52m46.656032092s) [noise=true , address=77.90.31.11:4553 , piece_id=FJJZYD55ZYEU7SMIY6YUJG7NETWWZSDAWQ7MKLCF7WZKAJD6PSDQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885562756 (561h52m46.656204081s) [noise=true , address=104.36.82.13:4187 , piece_id=D32HIFFGXLJGPNJ3MWITHYBHY6PJMPROYNY24WC2MLSKGYETEISQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885696472 (561h52m46.656337797s) [noise=true , address=104.36.82.5:4840 , piece_id=6NH4QMQKGIIPBIX2KWJVOBAO3KBD5NLBR3TU5WLTLQTT5B5LRA6A, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.885827152 (561h52m46.656468477s) [noise=true , address=77.90.31.13:4627 , piece_id=HO7HTXMXQIL5VQYZ6QW4JT32YS3FYDDC4P6OQM4YPOZETW2PH6TQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886076038 (561h52m46.656717373s) [noise=true , address=104.36.82.15:4258 , piece_id=KQXN75BSJBDZ5ADQ6DEIHGXHABYH4XTI3BWNYT4GP2JKTNIG65AQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886276822 (561h52m46.656918157s) [noise=true , address=77.90.31.17:4770 , piece_id=E5KK5NPBX7R2DDZHBFXJXMUW7NVJH5EMXC5P6PV4GMJK3YTNV4FQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886462738 (561h52m46.657104073s) [noise=true , address=45.91.116.9:4261 , piece_id=KV547AY2I3O7NCOC5XD4YUTBSEUDXXHTVHL5MZYITA6MNRHBLLTA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886653002 (561h52m46.657294337s) [noise=true , address=77.90.31.18:4807 , piece_id=LCQZNCW4VIPB4AXGKUMHFFVWKAJMQOUA2E46AGRADDOO34X7SWLQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.886840391 (561h52m46.657481726s) [noise=true , address=104.36.82.12:4151 , piece_id=PSP3UE26YG2YDQYNGQWL3AE2ZWGTUUBI4JOU6F2AJJBR67YYP5QA, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:11.887087203 (561h52m46.657728538s) [noise=true , address=104.36.82.16:4295 , piece_id=BLQ5CYCBLNORM7SN2UWCVVJTY7BZQ3FDNFDCHA6YDNF3A6P62SDQ, seg_pos={0 0}, upload=449305]: Uploading piece... Jul 16 19:03:13.650420754 (561h52m48.421062089s) [upload=449305]: Next returned nil segment Jul 16 19:03:13.650442185 (561h52m48.42108352s) [upload=449305]: Waiting for error group managing segments... Jul 16 19:03:13.957490344 (561h52m48.72813168s) [noise=true , address=104.36.82.13:4187 , piece_id=D32HIFFGXLJGPNJ3MWITHYBHY6PJMPROYNY24WC2MLSKGYETEISQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:13.998011145 (561h52m48.76865248s) [noise=true , address=104.36.82.16:4295 , piece_id=BLQ5CYCBLNORM7SN2UWCVVJTY7BZQ3FDNFDCHA6YDNF3A6P62SDQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.033735749 (561h52m48.804377084s) [noise=true , address=104.36.82.15:4258 , piece_id=KQXN75BSJBDZ5ADQ6DEIHGXHABYH4XTI3BWNYT4GP2JKTNIG65AQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.043837948 (561h52m48.814479283s) [noise=true , address=45.91.116.6:4975 , piece_id=CCRBVJH3JJ2S4RWBACGHGPI3PLG3QXQRVRRD7PRRVWVXLBAD54NA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.056883559 (561h52m48.827524894s) [noise=true , address=45.91.116.11:4117 , piece_id=OLL5H6H7LI7KLHXJBKYHTKYJJBBAMBTUHFLVNFF7IGCBTLCKMEWQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.098272381 (561h52m48.868913716s) [noise=true , address=104.36.82.5:4840 , piece_id=6NH4QMQKGIIPBIX2KWJVOBAO3KBD5NLBR3TU5WLTLQTT5B5LRA6A, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.165604781 (561h52m48.936246116s) [noise=true , address=45.91.116.4:4938 , piece_id=XQ5QX4Y4KCAHAS5CU3TLOCTYYUCNANB3Q5RZLFW6ZTT4NOCSG5MQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.183514829 (561h52m48.954156164s) [noise=true , address=104.36.82.12:4151 , piece_id=PSP3UE26YG2YDQYNGQWL3AE2ZWGTUUBI4JOU6F2AJJBR67YYP5QA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:14.215594298 (561h52m48.986235633s) [noise=true , address=45.91.116.9:4261 , piece_id=KV547AY2I3O7NCOC5XD4YUTBSEUDXXHTVHL5MZYITA6MNRHBLLTA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: <nil> Jul 16 19:03:40.996425523 (561h53m15.767066858s) [noise=true , address=77.90.31.11:4553 , piece_id=FJJZYD55ZYEU7SMIY6YUJG7NETWWZSDAWQ7MKLCF7WZKAJD6PSDQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:03:40.996509955 (561h53m15.76715129s) [noise=true , address=77.90.31.12:4592 , piece_id=JRH475SGAPR6VWSR5LOBWD2MENEEP4VI4KRWA4EYCX6WPPZRXBPA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:03:40.996649712 (561h53m15.767291047s) [noise=true , address=77.90.31.16:4734 , piece_id=CQNZGGK22L3ASWB7B7JZVQFSGIQXSPSW7QTVFF2APBCRQTYUKKJA, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37472->77.90.31.16:4734: use of closed network connection Jul 16 19:03:40.996700199 (561h53m15.767341534s) [noise=true , address=77.90.31.18:4806 , piece_id=HIZ5UWUX2NKUTFPMEXBKQWVQJNMGUKXKW664E3KWQY3CGE2EGM7Q, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52946->77.90.31.18:4806: use of closed network connection Jul 16 19:03:40.996724826 (561h53m15.767366161s) [noise=true , address=77.90.31.17:4770 , piece_id=E5KK5NPBX7R2DDZHBFXJXMUW7NVJH5EMXC5P6PV4GMJK3YTNV4FQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36218->77.90.31.17:4770: use of closed network connection Jul 16 19:03:40.996768891 (561h53m15.767410216s) [noise=true , address=77.90.31.13:4627 , piece_id=HO7HTXMXQIL5VQYZ6QW4JT32YS3FYDDC4P6OQM4YPOZETW2PH6TQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58806->77.90.31.13:4627: use of closed network connection Jul 16 19:03:40.996809488 (561h53m15.767450893s) [noise=true , address=77.90.31.18:4807 , piece_id=LCQZNCW4VIPB4AXGKUMHFFVWKAJMQOUA2E46AGRADDOO34X7SWLQ, seg_pos={0 0}, upload=449305]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50972->77.90.31.18:4807: use of closed network connection Jul 16 19:03:40.996845206 (561h53m15.767486591s) [seg_pos={0 0}, upload=449305]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 16 19:03:41.230111736 (561h53m16.000753071s) [upload=449305]: Done uploading segments... Jul 16 19:03:41.230136874 (561h53m16.000778209s) [upload=449305]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration:* 30.235264039s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.012987683s FAIL (2025-07-16 19:02:54 UTC) orbiter warning: context deadline exceeded Duration: 6.01301211s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.830391655s FAIL (2025-07-16 19:02:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.830418146s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.05562902s FAIL (2025-07-16 19:01:49 UTC) orbiter warning: context deadline exceeded Duration: 6.055650722s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.84831904s FAIL (2025-07-16 19:01:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.848342915s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.623309081s FAIL (2025-07-16 19:01:10 UTC) orbiter warning: context deadline exceeded Duration: 6.623331423s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.832961998s FAIL (2025-07-16 19:00:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.832988508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.794824545s FAIL (2025-07-16 19:00:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.794870633s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.017803265s FAIL (2025-07-16 19:00:08 UTC) orbiter warning: context deadline exceeded Duration: 6.017828462s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.83248065s FAIL (2025-07-16 18:59:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.832497883s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.279198382s FAIL (2025-07-16 18:59:38 UTC) orbiter warning: context deadline exceeded Duration: 6.279219472s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.837267956s FAIL (2025-07-16 18:59:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.837295801s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.464226421s FAIL (2025-07-16 18:59:08 UTC) orbiter warning: context deadline exceeded Duration: 6.464270446s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.772657167s FAIL (2025-07-16 18:59:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.772683169s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.713060693s FAIL (2025-07-16 18:58:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.713085531s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.728809271s FAIL (2025-07-16 18:58:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.728838917s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.081143626s FAIL (2025-07-16 18:57:56 UTC) orbiter warning: context deadline exceeded Duration: 6.081162963s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.800155199s FAIL (2025-07-16 18:57:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.80018141s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.668369893s FAIL (2025-07-16 18:57:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.66839435s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.200282575s FAIL (2025-07-16 18:57:07 UTC) orbiter warning: context deadline exceeded Duration: 6.200304707s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.219781391s FAIL (2025-07-16 18:56:36 UTC) orbiter warning: context deadline exceeded Duration: 6.219801959s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.737127691s FAIL (2025-07-16 18:56:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.737147908s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.308025828s FAIL (2025-07-16 18:55:39 UTC) orbiter warning: context deadline exceeded Duration: 6.308051506s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.910233246s FAIL (2025-07-16 18:55:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.910257373s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.83365907s FAIL (2025-07-16 18:55:02 UTC) orbiter warning: context deadline exceeded Duration: 5.833705188s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.829562861s FAIL (2025-07-16 18:54:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.829589132s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.871292184s FAIL (2025-07-16 18:54:41 UTC) orbiter warning: context deadline exceeded Duration: 5.871319016s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.821502601s FAIL (2025-07-16 18:54:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.821527507s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.269475305s FAIL (2025-07-16 18:53:40 UTC) orbiter warning: context deadline exceeded Duration: 6.269499021s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.017795665s FAIL (2025-07-16 18:53:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.017822938s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.024612913s FAIL (2025-07-16 18:52:59 UTC) orbiter warning: context deadline exceeded Duration: 6.024633864s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.954073625s FAIL (2025-07-16 18:52:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.954099545s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.198596005s FAIL (2025-07-16 18:52:26 UTC) orbiter warning: context deadline exceeded Duration: 6.198619459s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.764627481s FAIL (2025-07-16 18:51:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.764659432s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.961382582s FAIL (2025-07-16 18:51:22 UTC) orbiter warning: context deadline exceeded Duration: 5.961422589s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.80923032s FAIL (2025-07-16 18:50:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.809253685s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.295923708s FAIL (2025-07-16 18:50:42 UTC) orbiter warning: context deadline exceeded Duration: 6.295943827s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.86450035s FAIL (2025-07-16 18:50:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.86452111s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.339824229s FAIL (2025-07-16 18:50:12 UTC) orbiter warning: context deadline exceeded Duration: 6.339993133s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.824994142s FAIL (2025-07-16 18:49:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jul 16 18:40:14.162346704 (561h29m48.932988039s) [upload=448998]: Done uploading segments... Jul 16 18:40:14.162364248 (561h29m48.933005583s) [upload=448998]: Upload finished. err: <nil> Jul 16 18:40:26.790061078 (561h30m1.560702403s) [upload=449004]: Starting upload Jul 16 18:40:26.790122086 (561h30m1.560763421s) [upload=449004]: Done starting upload Jul 16 18:40:26.790255701 (561h30m1.560897046s) [upload=449004]: Uploading segments... Jul 16 18:40:26.790358027 (561h30m1.560999362s) [upload=449004]: Got next segment. Inline: false Jul 16 18:40:26.790393765 (561h30m1.5610351s) [upload=449004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 18:40:26.790399216 (561h30m1.561040551s) [upload=449004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 18:40:27.024250342 (561h30m1.794891677s) [seg_pos={0 0}, upload=449004]: Begin upload segment... Jul 16 18:40:27.024289247 (561h30m1.794930572s) [seg_pos={0 0}, upload=449004]: Done begin upload segment. Jul 16 18:40:27.024309335 (561h30m1.79495066s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024354372 (561h30m1.794995707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024647202 (561h30m1.795288537s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024829300 (561h30m1.795470635s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.024893494 (561h30m1.795534829s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025182626 (561h30m1.795823961s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025429118 (561h30m1.796070443s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.025669789 (561h30m1.796311124s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026478125 (561h30m1.79711946s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.026742371 (561h30m1.797383706s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.027023589 (561h30m1.797664924s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.028864902 (561h30m1.799506237s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029216585 (561h30m1.79985791s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029521789 (561h30m1.800163124s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.029930290 (561h30m1.800571625s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:27.030254411 (561h30m1.800895746s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Uploading piece... Jul 16 18:40:28.767225210 (561h30m3.537866535s) [upload=449004]: Next returned nil segment Jul 16 18:40:28.767306556 (561h30m3.537947891s) [upload=449004]: Waiting for error group managing segments... Jul 16 18:40:29.085631721 (561h30m3.856273056s) [noise=true , address=45.91.116.4:4938 , piece_id=MH57FQJW7LSKPCVNLOYPCGV35CL6GRLIZUT27SI5Y3NTKNOS4KIQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.092662945 (561h30m3.86330428s) [noise=true , address=45.91.116.5:4297 , piece_id=LDOJ366N3RNGF7YBVLDWFH4VKG43QV2VCB74JSYYSZQFP3PGZGCA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.131839973 (561h30m3.902481318s) [noise=true , address=45.91.116.13:4155 , piece_id=SLYG2S6VQUTBDAABI47YPXL6UCDHD3SRIVVDEMLAUGLR63CLFSRQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.146887965 (561h30m3.9175293s) [noise=true , address=104.36.82.8:4969 , piece_id=SVZWKGIUM6MABUDFUR3CN5ZMHLE7GVHMRBF6SFEOJSLJOTSINK4A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.162142703 (561h30m3.932784038s) [noise=true , address=104.36.82.14:4221 , piece_id=ILPLYEJ45YVE3QVHRJBZAKN7EGFZTMASHOKFYG7Z74R6AVTIIN4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.169925444 (561h30m3.940566779s) [noise=true , address=104.36.82.12:4151 , piece_id=Z76CBNLX5Z42KX6Q6HOKBQ4W2ZYO7VF622LSO6QPBA7YUZN5YNVA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.288603616 (561h30m4.059244951s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMEQDTY2BN3KFHDXOHCS3WSQN2PJI5JJJ2NYZ75ISPFR6GHX2XZA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.304769627 (561h30m4.075410962s) [noise=true , address=45.91.116.9:4263 , piece_id=ZNOASUR47WVW6BXURSBTLHYSC42KEHXOJJLPSV3WRR2NESZ7OV7Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.359375276 (561h30m4.130016611s) [noise=true , address=45.91.116.12:4225 , piece_id=ZUIY35X44GRSOOFWSJZJLOGWSWTIF4TGZU4XG5CWEL7426WIM6BQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.371795961 (561h30m4.142437296s) [noise=true , address=104.36.82.16:4294 , piece_id=EBLMBQFJ37QXGJHMFTPGXGWRAMYXIZJVMHRJ6FAEY6EO5GS636GA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:29.571621466 (561h30m4.342262791s) [noise=true , address=45.91.116.10:4081 , piece_id=FEN5O7E65LQYVBV2DEL2PQZ7NJCFGVJ3HGWFIMDR5ZNQT6PCSDAA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:51.206870849 (561h30m25.977512204s) [noise=true , address=77.90.31.13:4627 , piece_id=XYH2VDMXJZFT2JGRZ6TMK64QJ5HDRBQADDBHLKVGCQ62HPI4I4WA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: <nil> Jul 16 18:40:56.145115448 (561h30m30.915756783s) [noise=true , address=77.90.31.10:4517 , piece_id=RA2EVKJIC2KXXBCHHO222B7R4ZXY2DLVZ273ZOZNZHSWCRE3KCWA, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41382->77.90.31.10:4517: use of closed network connection Jul 16 18:40:56.145372068 (561h30m30.916013404s) [noise=true , address=77.90.31.21:4879 , piece_id=I24V4OGTG27JFZOF36JN66IWEFUHN35TGFLEOCQBRPR3HDKNGTUQ, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45536->77.90.31.21:4879: use of closed network connection Jul 16 18:40:56.145410391 (561h30m30.916051726s) [noise=true , address=77.90.31.16:4735 , piece_id=4BZZLOFJQOOEDQOVIDWV4UCVJCSEWCONAEQD2YTJWWA7QTZCJM4Q, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57378->77.90.31.16:4735: use of closed network connection Jul 16 18:40:56.145436632 (561h30m30.916077967s) [noise=true , address=77.90.31.16:4734 , piece_id=KFKECQTO2RM5XY6SIYHIRDV27QMTN6BSQAPXUIXKJFX7JDY5CQ5A, seg_pos={0 0}, upload=449004]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58498->77.90.31.16:4734: use of closed network connection Jul 16 18:40:56.145457642 (561h30m30.916098977s) [seg_pos={0 0}, upload=449004]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 18:40:56.379144706 (561h30m31.149786041s) [upload=449004]: Done uploading segments... Jul 16 18:40:56.379169884 (561h30m31.149811219s) [upload=449004]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.825022076s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)