Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.651988341s FAIL (2025-07-07 12:04:42 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments... Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil> Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments... Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment... Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment. Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments... Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces. Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments... Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.652012096s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.841590899s FAIL (2025-07-07 12:04:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.841626566s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.346857155s FAIL (2025-07-07 12:04:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 2.346875621s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.041433364s FAIL (2025-07-07 12:03:55 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments... Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil> Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments... Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment... Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment. Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments... Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces. Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments... Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.041469312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.304424902s FAIL (2025-07-07 12:03:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 2.304452156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.003788609s FAIL (2025-07-07 12:03:18 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments... Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil> Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments... Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment... Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment. Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments... Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces. Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments... Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.003807053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m13.670091352s FAIL (2025-07-07 12:03:12 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments... Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil> Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments... Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment... Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment. Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece... Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments... Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces. Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil> Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments... Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m13.670115177s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.31895782s FAIL (2025-07-07 12:03:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 2.318986794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.947830383s FAIL (2025-07-07 12:02:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.947892213s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.855937066s FAIL (2025-07-07 12:01:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.855988955s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.91198799s FAIL (2025-07-07 12:01:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.912025753s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.885976245s FAIL (2025-07-07 12:00:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.885994951s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.904227814s FAIL (2025-07-07 12:00:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.904253994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.849357251s FAIL (2025-07-07 11:59:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.849384683s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.880461799s FAIL (2025-07-07 11:59:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.880488851s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.900805069s FAIL (2025-07-07 11:58:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.900825418s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.89192959s FAIL (2025-07-07 11:57:52 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.891974414s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.905734461s FAIL (2025-07-07 11:57:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.905759217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.859329719s FAIL (2025-07-07 11:56:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.859354768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.900780715s FAIL (2025-07-07 11:56:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.900811344s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.933643583s FAIL (2025-07-07 11:55:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.933669843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.898317626s FAIL (2025-07-07 11:54:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.898350069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.954848804s FAIL (2025-07-07 11:54:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.954897277s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.895341308s FAIL (2025-07-07 11:52:52 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.895360976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.940996553s FAIL (2025-07-07 11:51:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.941018445s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.864063737s FAIL (2025-07-07 11:50:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.864095057s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.068540066s FAIL (2025-07-07 11:49:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 2.068584321s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.924654368s FAIL (2025-07-07 11:49:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 1.924678363s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001192354s FAIL (2025-07-07 11:48:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil> Duration: 30.001212093s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 371.093702ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:21:25 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 371.117759ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [XX.............] duration: 374.079968ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:21:02 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 374.110696ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.XX............] duration: 400.87042ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:20:14 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 400.902041ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 602.172305ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:19:13 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 602.1911ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [..XX...........] duration: 376.578756ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:18:59 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 376.603775ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 617.694712ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:18:42 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 617.733877ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [...XX..........] duration: 401.238835ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:18:37 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 401.265194ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [....XX.........] duration: 360.268412ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:18:00 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 360.293138ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 631.535872ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:17:48 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 631.557122ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.....XX........] duration: 533.675146ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:17:31 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 533.715824ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [......XX.......] duration: 381.765522ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:16:50 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 381.795281ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 615.973033ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:16:48 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 616.00798ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.......XX......] duration: 802.816506ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:15:57 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 802.840111ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 614.052509ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:15:42 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 614.0793ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 635.455547ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:15:25 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 635.481718ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 641.884436ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:15:01 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 641.91174ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [........XX.....] duration: 354.567274ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:14:54 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 354.602032ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 714.195264ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:14:45 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 714.22023ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 635.54648ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:14:15 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 635.577279ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 657.549617ms remote: "tcp:3.109.243.18:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-07 08:13:41 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 188.166.225.170:55518->3.109.243.18:443: read: connection reset by peer Duration: 657.567082ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.........XX....] duration: 381.056596ms remote: "tcp:13.203.125.58:443" target: https://www.storj.io/ FAIL (2025-07-07 08:13:39 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 188.166.225.170:45820->13.203.125.58:443: read: connection reset by peer Duration: 381.082254ms core website

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)