Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.033672235s
FAIL (2025-07-14 12:13:44 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 2.033745947s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.926405182s
FAIL (2025-07-14 12:12:58 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.926431704s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.914388897s
FAIL (2025-07-14 12:12:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.914417241s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.988108528s
FAIL (2025-07-14 12:11:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.988134678s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.870577787s
FAIL (2025-07-14 12:11:30 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.870604778s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.944145563s
FAIL (2025-07-14 12:10:45 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.944181972s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
1.899473611s
FAIL (2025-07-14 12:09:39 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.89949412s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.855205648s
FAIL (2025-07-14 12:09:15 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.855227871s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.866334111s
FAIL (2025-07-14 12:08:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.866353248s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.902717562s
FAIL (2025-07-14 12:07:36 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.90274732s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.933033319s
FAIL (2025-07-14 12:07:18 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.933062965s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
1.865759367s
FAIL (2025-07-14 12:06:55 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.865779646s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
1.889613054s
FAIL (2025-07-14 12:05:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.889640357s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.937282813s
FAIL (2025-07-14 12:04:42 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.937310215s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.909094321s
FAIL (2025-07-14 12:03:30 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.909130441s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.914872437s
FAIL (2025-07-14 12:03:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.914891013s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.915454156s
FAIL (2025-07-14 12:02:43 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.915481439s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.928705681s
FAIL (2025-07-14 12:02:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.928733837s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.92611249s
FAIL (2025-07-14 12:01:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.926149552s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.997315479s
FAIL (2025-07-14 12:00:58 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.997346267s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.869940313s
FAIL (2025-07-14 12:00:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.869975932s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.867404396s
FAIL (2025-07-14 11:59:43 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.867428762s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.113115767s
FAIL (2025-07-14 11:59:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 2.11314376s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.91890354s
FAIL (2025-07-14 11:58:18 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.918931473s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.897979301s
FAIL (2025-07-14 11:57:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.898004889s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.899221082s
FAIL (2025-07-14 11:57:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.899247593s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.923971112s
FAIL (2025-07-14 11:56:20 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.923996491s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.940831619s
FAIL (2025-07-14 11:55:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.940859051s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.854832935s
FAIL (2025-07-14 11:55:26 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.854853194s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.921019835s
FAIL (2025-07-14 11:55:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.921047737s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.928333731s
FAIL (2025-07-14 11:54:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.928358648s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.010882536s
FAIL (2025-07-14 11:54:28 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 2.010914427s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.878988036s
FAIL (2025-07-14 11:53:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.879015588s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.890684003s
FAIL (2025-07-14 11:53:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.890717697s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.248285682s
FAIL (2025-07-14 11:52:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 2.248309769s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.082745176s
FAIL (2025-07-14 11:52:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 2.082770002s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.926950768s
FAIL (2025-07-14 11:51:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.926985175s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.891019293s
FAIL (2025-07-14 11:50:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.8910748s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.895396748s
FAIL (2025-07-14 11:50:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.895425994s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.927336985s
FAIL (2025-07-14 11:48:52 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.92736553s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.921909963s
FAIL (2025-07-14 11:47:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.92192895s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.86147506s
FAIL (2025-07-14 11:46:44 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.861505267s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.893287626s
FAIL (2025-07-14 11:46:26 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.893361777s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.907064751s
FAIL (2025-07-14 11:46:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.907092946s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.935522239s
FAIL (2025-07-14 11:45:44 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.935547818s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.851142645s
FAIL (2025-07-14 11:45:21 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 1.851170617s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001121581s
FAIL (2025-07-14 11:45:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d=405126]: Starting upload Jul 14 11:36:33.929835737 (506h26m8.700477072s) [upload=405126]: Done starting upload Jul 14 11:36:33.929911632 (506h26m8.700552967s) [upload=405126]: Uploading segments... Jul 14 11:36:33.930008497 (506h26m8.700649842s) [upload=405126]: Got next segment. Inline: true Jul 14 11:36:33.930013988 (506h26m8.700655323s) [upload=405126]: Waiting for error group managing segments... Jul 14 11:36:33.930057200 (506h26m8.700698535s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:36:33.930062220 (506h26m8.700703555s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:36:33.930064714 (506h26m8.700706049s) [upload=405126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:36:34.206072311 (506h26m8.976713646s) [upload=405126]: Done uploading segments... Jul 14 11:36:34.206090204 (506h26m8.976731539s) [upload=405126]: Upload finished. err: <nil> Jul 14 11:37:32.626654418 (506h27m7.397295753s) [upload=405136]: Starting upload Jul 14 11:37:32.626704013 (506h27m7.397345378s) [upload=405136]: Done starting upload Jul 14 11:37:32.626780268 (506h27m7.397421603s) [upload=405136]: Uploading segments... Jul 14 11:37:32.626823241 (506h27m7.397464576s) [upload=405136]: Got next segment. Inline: true Jul 14 11:37:32.626827328 (506h27m7.397468663s) [upload=405136]: Waiting for error group managing segments... Jul 14 11:37:32.626870521 (506h27m7.397511867s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:37:32.626877895 (506h27m7.39751922s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:37:32.626880380 (506h27m7.397521715s) [upload=405136]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:37:32.901294748 (506h27m7.671936083s) [upload=405136]: Done uploading segments... Jul 14 11:37:32.901311050 (506h27m7.671952385s) [upload=405136]: Upload finished. err: <nil> Jul 14 11:38:34.033913720 (506h28m8.804555055s) [upload=405148]: Starting upload Jul 14 11:38:34.033998672 (506h28m8.804640007s) [upload=405148]: Done starting upload Jul 14 11:38:34.034065850 (506h28m8.804707185s) [upload=405148]: Uploading segments... Jul 14 11:38:34.034108993 (506h28m8.804750328s) [upload=405148]: Got next segment. Inline: true Jul 14 11:38:34.034114464 (506h28m8.804755829s) [upload=405148]: Waiting for error group managing segments... Jul 14 11:38:34.034160442 (506h28m8.804801787s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:38:34.034166473 (506h28m8.804807808s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:38:34.034170130 (506h28m8.804811485s) [upload=405148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:38:34.307942291 (506h28m9.078583626s) [upload=405148]: Done uploading segments... Jul 14 11:38:34.307960936 (506h28m9.078602271s) [upload=405148]: Upload finished. err: <nil> Jul 14 11:39:03.676579911 (506h28m38.447221247s) [upload=405157]: Starting upload Jul 14 11:39:03.676708348 (506h28m38.447349683s) [upload=405157]: Done starting upload Jul 14 11:39:03.676859146 (506h28m38.447500521s) [upload=405157]: Uploading segments... Jul 14 11:39:03.676909833 (506h28m38.447551188s) [upload=405157]: Got next segment. Inline: true Jul 14 11:39:03.676916586 (506h28m38.447557951s) [upload=405157]: Waiting for error group managing segments... Jul 14 11:39:03.676976992 (506h28m38.447618337s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:03.676985157 (506h28m38.447626522s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:03.676990397 (506h28m38.447631752s) [upload=405157]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:39:04.008029353 (506h28m38.778670688s) [upload=405157]: Done uploading segments... Jul 14 11:39:04.008094899 (506h28m38.778736234s) [upload=405157]: Upload finished. err: <nil> Jul 14 11:39:59.972968998 (506h29m34.743610333s) [upload=405169]: Starting upload Jul 14 11:39:59.973029022 (506h29m34.743670347s) [upload=405169]: Done starting upload Jul 14 11:39:59.973080982 (506h29m34.743722317s) [upload=405169]: Uploading segments... Jul 14 11:39:59.973110508 (506h29m34.743751843s) [upload=405169]: Got next segment. Inline: true Jul 14 11:39:59.973114756 (506h29m34.743756091s) [upload=405169]: Waiting for error group managing segments... Jul 14 11:39:59.973149653 (506h29m34.743790988s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:39:59.973154102 (506h29m34.743795427s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:39:59.973156557 (506h29m34.743797902s) [upload=405169]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:00.265946429 (506h29m35.036587764s) [upload=405169]: Done uploading segments... Jul 14 11:40:00.265964754 (506h29m35.036606089s) [upload=405169]: Upload finished. err: <nil> Jul 14 11:40:51.801421954 (506h30m26.572063289s) [upload=405182]: Starting upload Jul 14 11:40:51.801503651 (506h30m26.572144986s) [upload=405182]: Done starting upload Jul 14 11:40:51.801557754 (506h30m26.572199089s) [upload=405182]: Uploading segments... Jul 14 11:40:51.801589715 (506h30m26.57223105s) [upload=405182]: Got next segment. Inline: true Jul 14 11:40:51.801593553 (506h30m26.572234888s) [upload=405182]: Waiting for error group managing segments... Jul 14 11:40:51.801660862 (506h30m26.572302197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:40:51.801667114 (506h30m26.572308449s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:40:51.801671862 (506h30m26.572313197s) [upload=405182]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:40:52.090230801 (506h30m26.860872136s) [upload=405182]: Done uploading segments... Jul 14 11:40:52.090249095 (506h30m26.86089043s) [upload=405182]: Upload finished. err: <nil> Jul 14 11:41:34.623680150 (506h31m9.394321485s) [upload=405190]: Starting upload Jul 14 11:41:34.623741216 (506h31m9.394382541s) [upload=405190]: Done starting upload Jul 14 11:41:34.623805539 (506h31m9.394446874s) [upload=405190]: Uploading segments... Jul 14 11:41:34.623842130 (506h31m9.394483455s) [upload=405190]: Got next segment. Inline: true Jul 14 11:41:34.623846328 (506h31m9.394487653s) [upload=405190]: Waiting for error group managing segments... Jul 14 11:41:34.623881344 (506h31m9.394522669s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:34.623885883 (506h31m9.394527218s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:34.623888718 (506h31m9.394530053s) [upload=405190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:34.908640753 (506h31m9.679282088s) [upload=405190]: Done uploading segments... Jul 14 11:41:34.908659239 (506h31m9.679300574s) [upload=405190]: Upload finished. err: <nil> Jul 14 11:41:56.984724782 (506h31m31.755366117s) [upload=405199]: Starting upload Jul 14 11:41:56.984792691 (506h31m31.755434026s) [upload=405199]: Done starting upload Jul 14 11:41:56.984841725 (506h31m31.75548306s) [upload=405199]: Uploading segments... Jul 14 11:41:56.984871282 (506h31m31.755512617s) [upload=405199]: Got next segment. Inline: true Jul 14 11:41:56.984875239 (506h31m31.755516614s) [upload=405199]: Waiting for error group managing segments... Jul 14 11:41:56.984923892 (506h31m31.755565238s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:41:56.984930675 (506h31m31.75557202s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:41:56.984936307 (506h31m31.755577652s) [upload=405199]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:41:57.278894043 (506h31m32.049535378s) [upload=405199]: Done uploading segments... Jul 14 11:41:57.278912549 (506h31m32.049553884s) [upload=405199]: Upload finished. err: <nil> Jul 14 11:42:38.656670540 (506h32m13.427311875s) [upload=405206]: Starting upload Jul 14 11:42:38.656740694 (506h32m13.427382029s) [upload=405206]: Done starting upload Jul 14 11:42:38.656796831 (506h32m13.427438166s) [upload=405206]: Uploading segments... Jul 14 11:42:38.656827129 (506h32m13.427468454s) [upload=405206]: Got next segment. Inline: true Jul 14 11:42:38.656830546 (506h32m13.427471881s) [upload=405206]: Waiting for error group managing segments... Jul 14 11:42:38.656860995 (506h32m13.42750233s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:42:38.656865724 (506h32m13.427507059s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:42:38.656868990 (506h32m13.427510315s) [upload=405206]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:42:39.016758042 (506h32m13.787399377s) [upload=405206]: Done uploading segments... Jul 14 11:42:39.016776207 (506h32m13.787417542s) [upload=405206]: Upload finished. err: <nil> Jul 14 11:43:21.341133767 (506h32m56.111775102s) [upload=405218]: Starting upload Jul 14 11:43:21.341234849 (506h32m56.111876195s) [upload=405218]: Done starting upload Jul 14 11:43:21.341288933 (506h32m56.111930268s) [upload=405218]: Uploading segments... Jul 14 11:43:21.341319461 (506h32m56.111960796s) [upload=405218]: Got next segment. Inline: true Jul 14 11:43:21.341322637 (506h32m56.111963972s) [upload=405218]: Waiting for error group managing segments... Jul 14 11:43:21.341354107 (506h32m56.111995443s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 11:43:21.341360079 (506h32m56.112001415s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 11:43:21.341362594 (506h32m56.112003929s) [upload=405218]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 11:43:21.617196427 (506h32m56.387837763s) [upload=405218]: Done uploading segments... Jul 14 11:43:21.617211977 (506h32m56.387853312s) [upload=405218]: Upload finished. err: <nil>
Duration: 30.00114751s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
6.196148042s
FAIL (2025-07-14 11:35:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.196171887s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.36148454s
FAIL (2025-07-14 11:35:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.36150548s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.364519396s
FAIL (2025-07-14 11:34:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.364545765s
edge
gateway
us-select-1
|