Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.098661241s
FAIL (2025-07-07 12:27:23 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.098684093s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.896202123s
FAIL (2025-07-07 12:26:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.896223723s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.625914265s
FAIL (2025-07-07 12:26:46 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.625942008s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.89831209s
FAIL (2025-07-07 12:26:26 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.898341907s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.896041446s
FAIL (2025-07-07 12:26:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.896069559s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.735940403s
FAIL (2025-07-07 12:25:52 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.735964137s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.883206379s
FAIL (2025-07-07 12:24:54 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.883252076s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.692157069s
FAIL (2025-07-07 12:24:51 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.69218865s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.613269965s
FAIL (2025-07-07 12:24:23 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.613294953s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.09179208s
FAIL (2025-07-07 12:24:17 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.091817919s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.612396814s
FAIL (2025-07-07 12:23:45 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.612425719s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.0403442s
FAIL (2025-07-07 12:23:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.040366111s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.60110947s
FAIL (2025-07-07 12:23:02 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.601138948s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.608538934s
FAIL (2025-07-07 12:22:31 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.608564784s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.8526711s
FAIL (2025-07-07 12:22:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.852694936s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.611772841s
FAIL (2025-07-07 12:22:10 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.611811634s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.85174608s
FAIL (2025-07-07 12:21:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.85177693s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.635557344s
FAIL (2025-07-07 12:21:30 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.635588073s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.863229368s
FAIL (2025-07-07 12:21:16 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.86327717s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.809022764s
FAIL (2025-07-07 12:21:14 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.809046229s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.813769212s
FAIL (2025-07-07 12:20:24 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.813818687s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.953478652s
FAIL (2025-07-07 12:20:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.953505073s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.996967394s
FAIL (2025-07-07 12:20:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.996998522s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.601111434s
FAIL (2025-07-07 12:19:46 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.601137654s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.869975997s
FAIL (2025-07-07 12:18:52 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.870001005s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
5.675884348s
FAIL (2025-07-07 12:18:37 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.675909905s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.067130548s
FAIL (2025-07-07 12:17:57 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.06715336s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.928471638s
FAIL (2025-07-07 12:17:41 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.928528665s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.925387783s
FAIL (2025-07-07 12:17:23 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.925418581s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
1.958562399s
FAIL (2025-07-07 12:17:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.958594731s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.885069812s
FAIL (2025-07-07 12:16:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.885099488s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.609384872s
FAIL (2025-07-07 12:16:10 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.609413899s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.860161968s
FAIL (2025-07-07 12:15:55 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.860193408s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.816607557s
FAIL (2025-07-07 12:15:37 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.816633787s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.651767688s
FAIL (2025-07-07 12:15:15 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.651794711s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.795405165s
FAIL (2025-07-07 12:14:54 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.795429262s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.941198886s
FAIL (2025-07-07 12:14:42 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.941257728s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.611508847s
FAIL (2025-07-07 12:14:27 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.61153614s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.620293201s
FAIL (2025-07-07 12:14:12 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.620333066s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.177128561s
FAIL (2025-07-07 12:13:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.177162637s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.72063909s
FAIL (2025-07-07 12:13:31 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.720682022s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
1.961077398s
FAIL (2025-07-07 12:13:13 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.961105202s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.595071609s
FAIL (2025-07-07 12:12:52 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.595100314s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.295276852s
FAIL (2025-07-07 12:12:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 2.295301218s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.853934283s
FAIL (2025-07-07 12:12:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.853961155s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.790724728s
FAIL (2025-07-07 12:11:38 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.790755246s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.808389122s
FAIL (2025-07-07 12:11:09 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.80842981s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.919196902s
FAIL (2025-07-07 12:11:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.919246206s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.924636356s
FAIL (2025-07-07 12:10:25 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== upload Jul 7 11:40:35.936449645 (338h30m10.70709098s) [upload=270785]: Done starting upload Jul 7 11:40:35.936514559 (338h30m10.707155884s) [upload=270785]: Uploading segments... Jul 7 11:40:35.936545388 (338h30m10.707186713s) [upload=270785]: Got next segment. Inline: true Jul 7 11:40:35.936549045 (338h30m10.70719038s) [upload=270785]: Waiting for error group managing segments... Jul 7 11:40:35.936590585 (338h30m10.70723191s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:40:35.936595765 (338h30m10.7072371s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:40:35.936598359 (338h30m10.707239695s) [upload=270785]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:40:36.219622818 (338h30m10.990264153s) [upload=270785]: Done uploading segments... Jul 7 11:40:36.219640883 (338h30m10.990282218s) [upload=270785]: Upload finished. err: <nil> Jul 7 11:41:13.707744986 (338h30m48.478386321s) [upload=270792]: Starting upload Jul 7 11:41:13.707827083 (338h30m48.478468428s) [upload=270792]: Done starting upload Jul 7 11:41:13.707902237 (338h30m48.478543572s) [upload=270792]: Uploading segments... Jul 7 11:41:13.707936983 (338h30m48.478578318s) [upload=270792]: Got next segment. Inline: true Jul 7 11:41:13.707942153 (338h30m48.478583488s) [upload=270792]: Waiting for error group managing segments... Jul 7 11:41:13.707983392 (338h30m48.478624727s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:41:13.707988842 (338h30m48.478630188s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:41:13.707992089 (338h30m48.478633424s) [upload=270792]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:41:14.001490555 (338h30m48.77213191s) [upload=270792]: Done uploading segments... Jul 7 11:41:14.001513128 (338h30m48.772154474s) [upload=270792]: Upload finished. err: <nil> Jul 7 11:42:09.601159074 (338h31m44.371800399s) [upload=270808]: Starting upload Jul 7 11:42:09.601235320 (338h31m44.371876655s) [upload=270808]: Done starting upload Jul 7 11:42:09.601283062 (338h31m44.371924397s) [upload=270808]: Uploading segments... Jul 7 11:42:09.601340402 (338h31m44.371981737s) [upload=270808]: Got next segment. Inline: true Jul 7 11:42:09.601344690 (338h31m44.371986025s) [upload=270808]: Waiting for error group managing segments... Jul 7 11:42:09.601379576 (338h31m44.372020911s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:09.601384115 (338h31m44.37202545s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:09.601386520 (338h31m44.372027855s) [upload=270808]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:09.874882919 (338h31m44.645524264s) [upload=270808]: Done uploading segments... Jul 7 11:42:09.874905022 (338h31m44.645546377s) [upload=270808]: Upload finished. err: <nil> Jul 7 11:42:45.937793832 (338h32m20.708435167s) [upload=270814]: Starting upload Jul 7 11:42:45.937872012 (338h32m20.708513347s) [upload=270814]: Done starting upload Jul 7 11:42:45.937937177 (338h32m20.708578512s) [upload=270814]: Uploading segments... Jul 7 11:42:45.937977414 (338h32m20.708618749s) [upload=270814]: Got next segment. Inline: true Jul 7 11:42:45.937981401 (338h32m20.708622736s) [upload=270814]: Waiting for error group managing segments... Jul 7 11:42:45.938014504 (338h32m20.708655839s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:42:45.938019985 (338h32m20.70866132s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:42:45.938022731 (338h32m20.708664066s) [upload=270814]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:42:46.265957179 (338h32m21.036598504s) [upload=270814]: Done uploading segments... Jul 7 11:42:46.265975775 (338h32m21.0366171s) [upload=270814]: Upload finished. err: <nil> Jul 7 11:43:13.189081597 (338h32m47.959722932s) [upload=270822]: Starting upload Jul 7 11:43:13.189130460 (338h32m47.959771785s) [upload=270822]: Done starting upload Jul 7 11:43:13.189182450 (338h32m47.959823775s) [upload=270822]: Uploading segments... Jul 7 11:43:13.189420435 (338h32m47.96006177s) [upload=270822]: Got next segment. Inline: true Jul 7 11:43:13.189426627 (338h32m47.960067962s) [upload=270822]: Waiting for error group managing segments... Jul 7 11:43:13.189476563 (338h32m47.960117898s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:43:13.189481231 (338h32m47.960122556s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:43:13.189489096 (338h32m47.960130431s) [upload=270822]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:43:13.479015434 (338h32m48.249656769s) [upload=270822]: Done uploading segments... Jul 7 11:43:13.479032757 (338h32m48.249674082s) [upload=270822]: Upload finished. err: <nil> Jul 7 11:44:09.100465688 (338h33m43.871107023s) [upload=270833]: Starting upload Jul 7 11:44:09.100520703 (338h33m43.871162068s) [upload=270833]: Done starting upload Jul 7 11:44:09.100579906 (338h33m43.871221231s) [upload=270833]: Uploading segments... Jul 7 11:44:09.100613181 (338h33m43.871254516s) [upload=270833]: Got next segment. Inline: true Jul 7 11:44:09.100616577 (338h33m43.871257912s) [upload=270833]: Waiting for error group managing segments... Jul 7 11:44:09.100647586 (338h33m43.871288921s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:09.100651754 (338h33m43.871293089s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:09.100655080 (338h33m43.871296416s) [upload=270833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:09.377639331 (338h33m44.148280666s) [upload=270833]: Done uploading segments... Jul 7 11:44:09.377664910 (338h33m44.148306295s) [upload=270833]: Upload finished. err: <nil> Jul 7 11:44:50.210367722 (338h34m24.981009087s) [upload=270842]: Starting upload Jul 7 11:44:50.210511898 (338h34m24.981153303s) [upload=270842]: Done starting upload Jul 7 11:44:50.210592782 (338h34m24.981234128s) [upload=270842]: Uploading segments... Jul 7 11:44:50.210670622 (338h34m24.981311957s) [upload=270842]: Got next segment. Inline: true Jul 7 11:44:50.210675871 (338h34m24.981317207s) [upload=270842]: Waiting for error group managing segments... Jul 7 11:44:50.210711790 (338h34m24.981353125s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:44:50.210716409 (338h34m24.981357734s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:44:50.210718663 (338h34m24.981359998s) [upload=270842]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:44:50.487904509 (338h34m25.258545844s) [upload=270842]: Done uploading segments... Jul 7 11:44:50.487923305 (338h34m25.25856463s) [upload=270842]: Upload finished. err: <nil> Jul 7 11:45:39.475860963 (338h35m14.246502298s) [upload=270857]: Starting upload Jul 7 11:45:39.475917571 (338h35m14.246558906s) [upload=270857]: Done starting upload Jul 7 11:45:39.476027511 (338h35m14.246668846s) [upload=270857]: Uploading segments... Jul 7 11:45:39.476102966 (338h35m14.246744301s) [upload=270857]: Got next segment. Inline: true Jul 7 11:45:39.476107574 (338h35m14.246748909s) [upload=270857]: Waiting for error group managing segments... Jul 7 11:45:39.476151248 (338h35m14.246792583s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:45:39.476156288 (338h35m14.246797623s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:45:39.476158742 (338h35m14.246800078s) [upload=270857]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:45:39.753124308 (338h35m14.523765643s) [upload=270857]: Done uploading segments... Jul 7 11:45:39.753145458 (338h35m14.523786793s) [upload=270857]: Upload finished. err: <nil> Jul 7 11:46:23.856413065 (338h35m58.6270544s) [upload=270866]: Starting upload Jul 7 11:46:23.856472799 (338h35m58.627114134s) [upload=270866]: Done starting upload Jul 7 11:46:23.856530038 (338h35m58.627171374s) [upload=270866]: Uploading segments... Jul 7 11:46:23.856556740 (338h35m58.627198075s) [upload=270866]: Got next segment. Inline: true Jul 7 11:46:23.856560527 (338h35m58.627201862s) [upload=270866]: Waiting for error group managing segments... Jul 7 11:46:23.856596486 (338h35m58.627237821s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:46:23.856601255 (338h35m58.62724259s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:46:23.856603840 (338h35m58.627245175s) [upload=270866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:46:24.139310919 (338h35m58.909952254s) [upload=270866]: Done uploading segments... Jul 7 11:46:24.139332080 (338h35m58.909973405s) [upload=270866]: Upload finished. err: <nil> Jul 7 11:47:26.933829897 (338h37m1.704471232s) [upload=270880]: Starting upload Jul 7 11:47:26.933918737 (338h37m1.704560072s) [upload=270880]: Done starting upload Jul 7 11:47:26.933978712 (338h37m1.704620047s) [upload=270880]: Uploading segments... Jul 7 11:47:26.934031814 (338h37m1.704673189s) [upload=270880]: Got next segment. Inline: true Jul 7 11:47:26.934038416 (338h37m1.704679761s) [upload=270880]: Waiting for error group managing segments... Jul 7 11:47:26.934081408 (338h37m1.704722743s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 7 11:47:26.934087991 (338h37m1.704729336s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 7 11:47:26.934093762 (338h37m1.704735097s) [upload=270880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 7 11:47:27.218024752 (338h37m1.988666137s) [upload=270880]: Done uploading segments... Jul 7 11:47:27.218046784 (338h37m1.988688119s) [upload=270880]: Upload finished. err: <nil>
Duration: 1.92465953s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.720078238s
FAIL (2025-07-07 12:09:54 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
iece_id=76E2UENIVWN6TTA3YD57X6CPVI366A2SNYQX6HTSGVLG6M4CCAZA, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222432903 (338h50m57.993074238s) [noise=true , address=104.36.82.10:4078 , piece_id=RHOTQBNCH5MUDYCPQ37YGYX5BZNHTMO6BH43VCNYORRRU75RTWWQ, seg_pos={0 0}, upload=271062]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 7 12:01:23.222453401 (338h50m57.993094736s) [seg_pos={0 0}, upload=271062]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 7 12:01:23.222512745 (338h50m57.993154081s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:01:23.222516733 (338h50m57.993158068s) [upload=271062]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:01:23.524023810 (338h50m58.294665165s) [upload=271062]: Done uploading segments...
Jul 7 12:01:23.524047065 (338h50m58.29468841s) [upload=271062]: Upload finished. err: <nil>
Jul 7 12:01:59.027955412 (338h51m33.798596747s) [upload=271070]: Starting upload
Jul 7 12:01:59.028075351 (338h51m33.798716696s) [upload=271070]: Done starting upload
Jul 7 12:01:59.028139976 (338h51m33.798781311s) [upload=271070]: Uploading segments...
Jul 7 12:01:59.028299931 (338h51m33.798941266s) [upload=271070]: Got next segment. Inline: false
Jul 7 12:01:59.028416515 (338h51m33.79905785s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 7 12:01:59.028422416 (338h51m33.799063761s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 7 12:01:59.261880873 (338h51m34.032522218s) [seg_pos={0 0}, upload=271070]: Begin upload segment...
Jul 7 12:01:59.261985924 (338h51m34.03262726s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262067471 (338h51m34.032708816s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262498215 (338h51m34.03313955s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262520607 (338h51m34.033161952s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.262795914 (338h51m34.033437249s) [seg_pos={0 0}, upload=271070]: Done begin upload segment.
Jul 7 12:01:59.262857242 (338h51m34.033498577s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263419457 (338h51m34.034060792s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.263698150 (338h51m34.034339485s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264015678 (338h51m34.034657013s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264312575 (338h51m34.03495391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264672153 (338h51m34.035313488s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.264999710 (338h51m34.035641045s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265238698 (338h51m34.035880053s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.265691123 (338h51m34.036332468s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266330596 (338h51m34.036971931s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.266671569 (338h51m34.037312914s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:01:59.267012862 (338h51m34.037654207s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Uploading piece...
Jul 7 12:02:00.922180841 (338h51m35.692822176s) [upload=271070]: Next returned nil segment
Jul 7 12:02:00.922208914 (338h51m35.692850249s) [upload=271070]: Waiting for error group managing segments...
Jul 7 12:02:01.187253453 (338h51m35.957894788s) [noise=true , address=104.36.82.15:4259 , piece_id=MYFFPHWVFEKH6PPXJB5BYJZUCHPGIBOHTM6BNN4U3WGXVIHOD4YA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.251407842 (338h51m36.022049177s) [noise=true , address=104.36.82.15:4258 , piece_id=G7UJATYPZXGE7AA2EVASTD2QXUW2F5J3N2UXSEZF23ZF5GWQJMKQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.260774924 (338h51m36.031416259s) [noise=true , address=104.36.82.9:4005 , piece_id=TZ43ZHEMJNM6P4IPJMMARHLKEX2FHTBIEFBY43IB6AAAEJMDCQNA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.275440331 (338h51m36.046081666s) [noise=true , address=104.36.82.3:4767 , piece_id=C5FKFSEABS2TO7GGPXU4J44NHKHYP7KFJZA6HNHHOETWQ6PODG4Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.292891458 (338h51m36.063532813s) [noise=true , address=104.36.82.10:4077 , piece_id=ZPX674LYZ7BTO2MV7T2SXNMYXLDZJ7BC64DG2XDIFJTVYMTBFRGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.294071356 (338h51m36.064712701s) [noise=true , address=104.36.82.16:4294 , piece_id=CNFTWMXOAONQYFMJVQAYU4FQ6ZCYNVKHBXKJASI3KN7GDJOVSZ6Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.310479056 (338h51m36.081120391s) [noise=true , address=104.36.82.12:4150 , piece_id=JVWXXQE2KUCN7TAEJPMXNJ53EASYQSF7BDLET74FKJ42U4O54HLQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.319958452 (338h51m36.090599787s) [noise=true , address=104.36.82.11:4113 , piece_id=TRHOT32ONTK5AHYGBOKUWOPSLJSOEZHWM7I2BEF35B6I2CK62TKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.345635898 (338h51m36.116277243s) [noise=true , address=45.91.116.11:4118 , piece_id=OH3XLDA6VDWB66QSJ7R3Y5TTMNARBDXS7MPLFDFKNSP7QZX76EGQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.348221234 (338h51m36.118862569s) [noise=true , address=45.91.116.9:4262 , piece_id=NPP4NPDONFYKTDJQKC5XPJPNEKKLYRAPHPNE2WKI3KOKG3VJWEKA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379148724 (338h51m36.149790069s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXAIRV6KPBEKHHT2WHTV7RFBZZX2M42BSDX34IICONNVFD2H4MPA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.379665763 (338h51m36.150307098s) [noise=true , address=77.90.31.10:4517 , piece_id=3YDX4SEJC372T64RTATEJFPBJBNOVGWE5QME2L6N2MWVWUN2P7ZQ, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383550435 (338h51m36.15419178s) [noise=true , address=77.90.31.13:4626 , piece_id=FMDUEB4OUQSGOWWO4LL6XIGKZFKU6ZFJDY7UJPTPRBU57OZIPXSA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.383582847 (338h51m36.154224182s) [seg_pos={0 0}, upload=271070]: Segment reached optimal threshold of 13 pieces.
Jul 7 12:02:01.383717544 (338h51m36.15435888s) [noise=true , address=77.90.31.12:4592 , piece_id=3K4UALXEA6HXATI7MAEK3TF2BSKRS4FTORIKFQUWALIENJ7ROGZA, seg_pos={0 0}, upload=271070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF
Jul 7 12:02:01.385367622 (338h51m36.156008967s) [noise=true , address=77.90.31.18:4807 , piece_id=JSZN5WN4HQN2JGFIJYMVFPDELMH67OU2UBGBWN7EYZ723NRNGX2Q, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.385867197 (338h51m36.156508542s) [noise=true , address=104.36.82.14:4223 , piece_id=4SKO77JRLCKZIJOA2S2ED3DHHDX4N6POWUIMJSOQNJQRP5SBRP2A, seg_pos={0 0}, upload=271070]: Done uploading piece. err: <nil>
Jul 7 12:02:01.386478106 (338h51m36.157119451s) [seg_pos={0 0}, upload=271070]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 7 12:02:01.386554532 (338h51m36.157195867s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 7 12:02:01.386559752 (338h51m36.157201097s) [upload=271070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 7 12:03:12.051066824 (338h52m46.821708159s) [upload=271070]: Done uploading segments...
Jul 7 12:03:12.051087384 (338h52m46.821728719s) [upload=271070]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.720105049s
core
uplink
us-select-1
|