Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.876508335s
FAIL (2025-07-20 21:35:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.876534234s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.875760352s
FAIL (2025-07-20 21:34:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.875786533s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.885872962s
FAIL (2025-07-20 21:33:59 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.885899672s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.900439432s
FAIL (2025-07-20 21:32:46 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.900466103s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.95997866s
FAIL (2025-07-20 21:32:03 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.960008666s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.86911024s
FAIL (2025-07-20 21:31:40 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.869153063s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.875111172s
FAIL (2025-07-20 21:31:17 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.875154153s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.938147902s
FAIL (2025-07-20 21:30:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.9381786s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.889269941s
FAIL (2025-07-20 21:29:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.889289479s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.831332449s
FAIL (2025-07-20 21:28:39 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.831423994s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.861856462s
FAIL (2025-07-20 21:27:40 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.861883966s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.998103305s
FAIL (2025-07-20 21:26:42 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.998124714s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.85252013s
FAIL (2025-07-20 21:26:26 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.852543083s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.838655802s
FAIL (2025-07-20 21:25:45 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.838687272s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.005048043s
FAIL (2025-07-20 21:25:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 2.005075093s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.930415175s
FAIL (2025-07-20 21:24:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.930471592s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.908495966s
FAIL (2025-07-20 21:23:47 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.908526394s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.096651208s
FAIL (2025-07-20 21:22:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 2.096678339s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.036957394s
FAIL (2025-07-20 21:22:23 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 2.036976691s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.902678125s
FAIL (2025-07-20 21:21:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.902704216s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.022995248s
FAIL (2025-07-20 21:20:47 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 2.023020306s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.848552628s
FAIL (2025-07-20 21:20:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.848584497s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.893927553s
FAIL (2025-07-20 21:19:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.893952381s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.913274971s
FAIL (2025-07-20 21:19:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.913303587s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.845736118s
FAIL (2025-07-20 21:18:00 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.845770204s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.877318305s
FAIL (2025-07-20 21:17:37 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.877352631s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.91325904s
FAIL (2025-07-20 21:16:36 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.91328498s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.89676397s
FAIL (2025-07-20 21:15:52 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.896788759s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.879978564s
FAIL (2025-07-20 21:15:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.880010576s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.864585772s
FAIL (2025-07-20 21:14:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.864621069s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.832977116s
FAIL (2025-07-20 21:13:30 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 1.833003086s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.028188727s
FAIL (2025-07-20 21:12:33 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 2.028212342s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000956639s
FAIL (2025-07-20 21:12:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad=36912]: Done uploading segments... Jul 20 21:04:13.677519035 (46h13m20.05009839s) [upload=36912]: Upload finished. err: <nil> Jul 20 21:04:38.321894955 (46h13m44.69447431s) [upload=36917]: Starting upload Jul 20 21:04:38.321948858 (46h13m44.694528213s) [upload=36917]: Done starting upload Jul 20 21:04:38.321998483 (46h13m44.694577838s) [upload=36917]: Uploading segments... Jul 20 21:04:38.322030634 (46h13m44.694609979s) [upload=36917]: Got next segment. Inline: true Jul 20 21:04:38.322037026 (46h13m44.694616371s) [upload=36917]: Waiting for error group managing segments... Jul 20 21:04:38.322068807 (46h13m44.694648162s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:04:38.322073266 (46h13m44.694652621s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:04:38.322075671 (46h13m44.694655026s) [upload=36917]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:04:38.620420428 (46h13m44.992999783s) [upload=36917]: Done uploading segments... Jul 20 21:04:38.620437882 (46h13m44.993017237s) [upload=36917]: Upload finished. err: <nil> Jul 20 21:05:12.846004647 (46h14m19.218584002s) [upload=36923]: Starting upload Jul 20 21:05:12.846059702 (46h14m19.218639057s) [upload=36923]: Done starting upload Jul 20 21:05:12.846111621 (46h14m19.218690976s) [upload=36923]: Uploading segments... Jul 20 21:05:12.846147460 (46h14m19.218726815s) [upload=36923]: Got next segment. Inline: true Jul 20 21:05:12.846151006 (46h14m19.218730351s) [upload=36923]: Waiting for error group managing segments... Jul 20 21:05:12.846182256 (46h14m19.218761611s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:05:12.846186936 (46h14m19.218766291s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:05:12.846189631 (46h14m19.218768986s) [upload=36923]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:05:13.107295768 (46h14m19.479875123s) [upload=36923]: Done uploading segments... Jul 20 21:05:13.107315646 (46h14m19.479895001s) [upload=36923]: Upload finished. err: <nil> Jul 20 21:06:16.301788086 (46h15m22.674367441s) [upload=36939]: Starting upload Jul 20 21:06:16.301871996 (46h15m22.674451351s) [upload=36939]: Done starting upload Jul 20 21:06:16.301955386 (46h15m22.674534741s) [upload=36939]: Uploading segments... Jul 20 21:06:16.302046020 (46h15m22.674625375s) [upload=36939]: Got next segment. Inline: true Jul 20 21:06:16.302050829 (46h15m22.674630174s) [upload=36939]: Waiting for error group managing segments... Jul 20 21:06:16.302093029 (46h15m22.674672384s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:06:16.302098530 (46h15m22.674677885s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:06:16.302100815 (46h15m22.67468017s) [upload=36939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:06:16.574020181 (46h15m22.946599536s) [upload=36939]: Done uploading segments... Jul 20 21:06:16.574039678 (46h15m22.946619033s) [upload=36939]: Upload finished. err: <nil> Jul 20 21:07:05.296385791 (46h16m11.668965147s) [upload=36948]: Starting upload Jul 20 21:07:05.296439294 (46h16m11.669018649s) [upload=36948]: Done starting upload Jul 20 21:07:05.296491223 (46h16m11.669070578s) [upload=36948]: Uploading segments... Jul 20 21:07:05.296525328 (46h16m11.669104683s) [upload=36948]: Got next segment. Inline: true Jul 20 21:07:05.296529025 (46h16m11.66910838s) [upload=36948]: Waiting for error group managing segments... Jul 20 21:07:05.296565796 (46h16m11.669145151s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:05.296570906 (46h16m11.669150251s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:05.296573851 (46h16m11.669153196s) [upload=36948]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:05.573631929 (46h16m11.946211284s) [upload=36948]: Done uploading segments... Jul 20 21:07:05.573649023 (46h16m11.946228368s) [upload=36948]: Upload finished. err: <nil> Jul 20 21:07:56.373712750 (46h17m2.746292105s) [upload=36960]: Starting upload Jul 20 21:07:56.373783155 (46h17m2.74636251s) [upload=36960]: Done starting upload Jul 20 21:07:56.373851666 (46h17m2.746431011s) [upload=36960]: Uploading segments... Jul 20 21:07:56.373949193 (46h17m2.746528548s) [upload=36960]: Got next segment. Inline: true Jul 20 21:07:56.373953731 (46h17m2.746533086s) [upload=36960]: Waiting for error group managing segments... Jul 20 21:07:56.374014357 (46h17m2.746593712s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:07:56.374020318 (46h17m2.746599673s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:07:56.374025790 (46h17m2.746605155s) [upload=36960]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:07:56.652828505 (46h17m3.02540786s) [upload=36960]: Done uploading segments... Jul 20 21:07:56.652845779 (46h17m3.025425134s) [upload=36960]: Upload finished. err: <nil> Jul 20 21:09:02.925460116 (46h18m9.298039461s) [upload=36974]: Starting upload Jul 20 21:09:02.925517746 (46h18m9.298097101s) [upload=36974]: Done starting upload Jul 20 21:09:02.925566349 (46h18m9.298145704s) [upload=36974]: Uploading segments... Jul 20 21:09:02.925612427 (46h18m9.298191782s) [upload=36974]: Got next segment. Inline: true Jul 20 21:09:02.925617185 (46h18m9.298196531s) [upload=36974]: Waiting for error group managing segments... Jul 20 21:09:02.925653295 (46h18m9.29823265s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:02.925658585 (46h18m9.29823794s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:02.925661090 (46h18m9.298240445s) [upload=36974]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:03.198427337 (46h18m9.571006692s) [upload=36974]: Done uploading segments... Jul 20 21:09:03.198446344 (46h18m9.571025699s) [upload=36974]: Upload finished. err: <nil> Jul 20 21:09:50.518152044 (46h18m56.890731399s) [upload=36989]: Starting upload Jul 20 21:09:50.518213932 (46h18m56.890793287s) [upload=36989]: Done starting upload Jul 20 21:09:50.518271142 (46h18m56.890850497s) [upload=36989]: Uploading segments... Jul 20 21:09:50.518299306 (46h18m56.890878661s) [upload=36989]: Got next segment. Inline: true Jul 20 21:09:50.518302833 (46h18m56.890882188s) [upload=36989]: Waiting for error group managing segments... Jul 20 21:09:50.518337399 (46h18m56.890916754s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:09:50.518342077 (46h18m56.890921432s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:09:50.518346145 (46h18m56.8909255s) [upload=36989]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:09:50.793305337 (46h18m57.165884692s) [upload=36989]: Done uploading segments... Jul 20 21:09:50.793323522 (46h18m57.165902867s) [upload=36989]: Upload finished. err: <nil> Jul 20 21:10:06.910915760 (46h19m13.283495155s) [upload=36992]: Starting upload Jul 20 21:10:06.911047172 (46h19m13.283626547s) [upload=36992]: Done starting upload Jul 20 21:10:06.911137204 (46h19m13.283716559s) [upload=36992]: Uploading segments... Jul 20 21:10:06.911222206 (46h19m13.283801561s) [upload=36992]: Got next segment. Inline: true Jul 20 21:10:06.911230974 (46h19m13.283810329s) [upload=36992]: Waiting for error group managing segments... Jul 20 21:10:06.911275638 (46h19m13.283854994s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:06.911280938 (46h19m13.283860304s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:06.911284506 (46h19m13.283863861s) [upload=36992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:07.188023904 (46h19m13.560603259s) [upload=36992]: Done uploading segments... Jul 20 21:10:07.188046277 (46h19m13.560625642s) [upload=36992]: Upload finished. err: <nil> Jul 20 21:10:24.474001527 (46h19m30.846580882s) [upload=36997]: Starting upload Jul 20 21:10:24.474105496 (46h19m30.846684891s) [upload=36997]: Done starting upload Jul 20 21:10:24.474177664 (46h19m30.846757019s) [upload=36997]: Uploading segments... Jul 20 21:10:24.474217580 (46h19m30.846796935s) [upload=36997]: Got next segment. Inline: true Jul 20 21:10:24.474223562 (46h19m30.846802907s) [upload=36997]: Waiting for error group managing segments... Jul 20 21:10:24.474274910 (46h19m30.846854255s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:10:24.474281603 (46h19m30.846860968s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:10:24.474287164 (46h19m30.846866529s) [upload=36997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:10:24.742102808 (46h19m31.114682163s) [upload=36997]: Done uploading segments... Jul 20 21:10:24.742122797 (46h19m31.114702152s) [upload=36997]: Upload finished. err: <nil> Jul 20 21:11:24.345418141 (46h20m30.717997506s) [upload=37010]: Starting upload Jul 20 21:11:24.345503054 (46h20m30.718082419s) [upload=37010]: Done starting upload Jul 20 21:11:24.345563129 (46h20m30.718142484s) [upload=37010]: Uploading segments... Jul 20 21:11:24.345600551 (46h20m30.718179906s) [upload=37010]: Got next segment. Inline: true Jul 20 21:11:24.345605720 (46h20m30.718185075s) [upload=37010]: Waiting for error group managing segments... Jul 20 21:11:24.345648973 (46h20m30.718228328s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 20 21:11:24.345656998 (46h20m30.718236354s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 20 21:11:24.345659954 (46h20m30.718239309s) [upload=37010]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 20 21:11:24.644900284 (46h20m31.017479639s) [upload=37010]: Done uploading segments... Jul 20 21:11:24.644918277 (46h20m31.017497622s) [upload=37010]: Upload finished. err: <nil>
Duration: 30.000977269s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
177.491706ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:41:30 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 177.51382ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [XX..........................................................] duration:
171.123383ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:40:51 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 171.153731ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.XX.........................................................] duration:
229.70554ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:39:41 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 229.729425ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..XX........................................................] duration:
176.780207ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:38:48 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 176.842476ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...XX.......................................................] duration:
171.771864ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:37:46 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 171.794307ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [....XX......................................................] duration:
174.994339ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:37:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 175.015099ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.....XX.....................................................] duration:
179.746112ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:37:02 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 179.768776ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [......XX....................................................] duration:
171.334595ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:36:24 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 171.357619ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.......XX...................................................] duration:
172.949815ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:35:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 172.973659ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [........XX..................................................] duration:
181.39752ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:35:23 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 181.420064ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.........XX.................................................] duration:
181.045357ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:34:28 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 181.063943ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..........XX................................................] duration:
174.496117ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:33:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 174.511826ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...........XX...............................................] duration:
177.15792ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:32:27 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 177.178538ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [............XX..............................................] duration:
177.948099ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:32:10 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 177.970051ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.............XX.............................................] duration:
173.679283ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:31:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 173.694202ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..............XX............................................] duration:
174.89501ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:31:00 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 174.914898ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...............XX...........................................] duration:
204.991563ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-20 11:30:43 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 205.01586ms
edge
linksharing
bgp
eu1
|