Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [.......................................XX.XX..X.XX..........] duration:
236.310003ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 21:45:56 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 236.34453ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
31 out of 60 is failed (threshold=3) [.........................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..] duration:
1.99045264s
FAIL (2025-08-07 21:45:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 61314643s) [upload=12402]: Done starting upload Aug 7 21:35:31.372062443 (11h55m30.96140743s) [upload=12402]: Uploading segments... Aug 7 21:35:31.372108090 (11h55m30.961453077s) [upload=12402]: Got next segment. Inline: true Aug 7 21:35:31.372114412 (11h55m30.961459399s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:35:31.372166783 (11h55m30.96151177s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:31.372172002 (11h55m30.961516999s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:31.372174898 (11h55m30.961519885s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:30.721260917 (11h56m30.310605904s) [upload=12402]: Done uploading segments... Aug 7 21:36:30.721280625 (11h56m30.310625612s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:33.253873027 (11h56m32.843218014s) [upload=12416]: Starting upload Aug 7 21:36:33.253941188 (11h56m32.843286175s) [upload=12416]: Done starting upload Aug 7 21:36:33.254004108 (11h56m32.843349085s) [upload=12416]: Uploading segments... Aug 7 21:36:33.254039966 (11h56m32.843384953s) [upload=12416]: Got next segment. Inline: true Aug 7 21:36:33.254043462 (11h56m32.84338845s) [upload=12416]: Waiting for error group managing segments... Aug 7 21:36:33.254074321 (11h56m32.843419308s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:33.254078290 (11h56m32.843423277s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:33.254080714 (11h56m32.843425701s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:32.603532648 (11h57m32.192877635s) [upload=12416]: Done uploading segments... Aug 7 21:37:32.603556113 (11h57m32.1929011s) [upload=12416]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:33.470049892 (11h57m33.059394869s) [upload=12434]: Starting upload Aug 7 21:37:33.470104877 (11h57m33.059449865s) [upload=12434]: Done starting upload Aug 7 21:37:33.470145565 (11h57m33.059490552s) [upload=12434]: Uploading segments... Aug 7 21:37:33.470187375 (11h57m33.059532352s) [upload=12434]: Got next segment. Inline: true Aug 7 21:37:33.470190872 (11h57m33.059535859s) [upload=12434]: Waiting for error group managing segments... Aug 7 21:37:33.470223234 (11h57m33.059568221s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:33.470226710 (11h57m33.059571697s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:33.470228904 (11h57m33.059573881s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:32.827667037 (11h58m32.417012024s) [upload=12434]: Done uploading segments... Aug 7 21:38:32.827687857 (11h58m32.417032834s) [upload=12434]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:33.703666664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.990478688s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
36 out of 60 is failed (threshold=3) [.................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X......] duration:
2.413443248s
FAIL (2025-08-07 21:45:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.413465239s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
33 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X.] duration:
1.912236129s
FAIL (2025-08-07 21:45:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.912261176s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.....] duration:
6.370585817s
FAIL (2025-08-07 21:45:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.370600445s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
28 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXXXXXXXXXXXX..XX] duration:
30.00132443s
FAIL (2025-08-07 21:45:23 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001339369s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
31 out of 60 is failed (threshold=3) [..........................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX.] duration:
5.628785992s
FAIL (2025-08-07 21:45:16 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 61314643s) [upload=12402]: Done starting upload Aug 7 21:35:31.372062443 (11h55m30.96140743s) [upload=12402]: Uploading segments... Aug 7 21:35:31.372108090 (11h55m30.961453077s) [upload=12402]: Got next segment. Inline: true Aug 7 21:35:31.372114412 (11h55m30.961459399s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:35:31.372166783 (11h55m30.96151177s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:31.372172002 (11h55m30.961516999s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:31.372174898 (11h55m30.961519885s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:30.721260917 (11h56m30.310605904s) [upload=12402]: Done uploading segments... Aug 7 21:36:30.721280625 (11h56m30.310625612s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:33.253873027 (11h56m32.843218014s) [upload=12416]: Starting upload Aug 7 21:36:33.253941188 (11h56m32.843286175s) [upload=12416]: Done starting upload Aug 7 21:36:33.254004108 (11h56m32.843349085s) [upload=12416]: Uploading segments... Aug 7 21:36:33.254039966 (11h56m32.843384953s) [upload=12416]: Got next segment. Inline: true Aug 7 21:36:33.254043462 (11h56m32.84338845s) [upload=12416]: Waiting for error group managing segments... Aug 7 21:36:33.254074321 (11h56m32.843419308s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:33.254078290 (11h56m32.843423277s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:33.254080714 (11h56m32.843425701s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:32.603532648 (11h57m32.192877635s) [upload=12416]: Done uploading segments... Aug 7 21:37:32.603556113 (11h57m32.1929011s) [upload=12416]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:33.470049892 (11h57m33.059394869s) [upload=12434]: Starting upload Aug 7 21:37:33.470104877 (11h57m33.059449865s) [upload=12434]: Done starting upload Aug 7 21:37:33.470145565 (11h57m33.059490552s) [upload=12434]: Uploading segments... Aug 7 21:37:33.470187375 (11h57m33.059532352s) [upload=12434]: Got next segment. Inline: true Aug 7 21:37:33.470190872 (11h57m33.059535859s) [upload=12434]: Waiting for error group managing segments... Aug 7 21:37:33.470223234 (11h57m33.059568221s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:33.470226710 (11h57m33.059571697s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:33.470228904 (11h57m33.059573881s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:32.827667037 (11h58m32.417012024s) [upload=12434]: Done uploading segments... Aug 7 21:38:32.827687857 (11h58m32.417032834s) [upload=12434]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:33.703666664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.6288008s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........................X...............X..........X.....] duration:
2.303690984s
FAIL (2025-08-07 21:45:15 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil>
Duration: 2.303715221s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
31 out of 60 is failed (threshold=3) [...........................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX] duration:
1m0.000975414s
FAIL (2025-08-07 21:45:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 61314643s) [upload=12402]: Done starting upload Aug 7 21:35:31.372062443 (11h55m30.96140743s) [upload=12402]: Uploading segments... Aug 7 21:35:31.372108090 (11h55m30.961453077s) [upload=12402]: Got next segment. Inline: true Aug 7 21:35:31.372114412 (11h55m30.961459399s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:35:31.372166783 (11h55m30.96151177s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:31.372172002 (11h55m30.961516999s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:31.372174898 (11h55m30.961519885s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:30.721260917 (11h56m30.310605904s) [upload=12402]: Done uploading segments... Aug 7 21:36:30.721280625 (11h56m30.310625612s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:33.253873027 (11h56m32.843218014s) [upload=12416]: Starting upload Aug 7 21:36:33.253941188 (11h56m32.843286175s) [upload=12416]: Done starting upload Aug 7 21:36:33.254004108 (11h56m32.843349085s) [upload=12416]: Uploading segments... Aug 7 21:36:33.254039966 (11h56m32.843384953s) [upload=12416]: Got next segment. Inline: true Aug 7 21:36:33.254043462 (11h56m32.84338845s) [upload=12416]: Waiting for error group managing segments... Aug 7 21:36:33.254074321 (11h56m32.843419308s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:33.254078290 (11h56m32.843423277s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:33.254080714 (11h56m32.843425701s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:32.603532648 (11h57m32.192877635s) [upload=12416]: Done uploading segments... Aug 7 21:37:32.603556113 (11h57m32.1929011s) [upload=12416]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:33.470049892 (11h57m33.059394869s) [upload=12434]: Starting upload Aug 7 21:37:33.470104877 (11h57m33.059449865s) [upload=12434]: Done starting upload Aug 7 21:37:33.470145565 (11h57m33.059490552s) [upload=12434]: Uploading segments... Aug 7 21:37:33.470187375 (11h57m33.059532352s) [upload=12434]: Got next segment. Inline: true Aug 7 21:37:33.470190872 (11h57m33.059535859s) [upload=12434]: Waiting for error group managing segments... Aug 7 21:37:33.470223234 (11h57m33.059568221s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:33.470226710 (11h57m33.059571697s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:33.470228904 (11h57m33.059573881s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:32.827667037 (11h58m32.417012024s) [upload=12434]: Done uploading segments... Aug 7 21:38:32.827687857 (11h58m32.417032834s) [upload=12434]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:33.703666664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.000989472s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
27 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXXXXXXXXXXXX...X..] duration:
6.590385551s
FAIL (2025-08-07 21:45:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.590408907s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [........................................XX.XX..X.XX.........] duration:
238.397116ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 21:45:06 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 238.42549ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...............X.........X......X..X.............X..X.......] duration:
8.018277037s
FAIL (2025-08-07 21:45:05 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
ad=12478]: Done starting upload
Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments...
Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false
Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment...
Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment.
Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment
Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments...
Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments...
Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil>
Duration: 8.018305813s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
36 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.....] duration:
7.259611514s
FAIL (2025-08-07 21:45:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.259632193s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
33 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X] duration:
30.000994257s
FAIL (2025-08-07 21:44:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001014445s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
33 out of 60 is failed (threshold=3) [...................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X...] duration:
1.961346488s
FAIL (2025-08-07 21:44:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 1.961369774s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [..................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.] duration:
7.852119277s
FAIL (2025-08-07 21:44:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
21:43:27.129330933 (12h3m26.71867592s) [upload=12525]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:43:27.998435988 (12h3m27.587780976s) [upload=12533]: Starting upload
Aug 7 21:43:27.998485233 (12h3m27.58783022s) [upload=12533]: Done starting upload
Aug 7 21:43:27.998649527 (12h3m27.587994504s) [upload=12533]: Uploading segments...
Aug 7 21:43:27.998680697 (12h3m27.588025674s) [upload=12533]: Got next segment. Inline: false
Aug 7 21:43:27.998702498 (12h3m27.588047475s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:43:27.998706827 (12h3m27.588051804s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:43:30.740636996 (12h3m30.329981983s) [seg_pos={0 0}, upload=12533]: Begin upload segment...
Aug 7 21:43:30.740675639 (12h3m30.330020626s) [seg_pos={0 0}, upload=12533]: Done begin upload segment.
Aug 7 21:43:30.740696189 (12h3m30.330041176s) [noise=true , address=45.91.116.11:4119 , piece_id=MEWVLFIXGSXTBBYJEYOJQVQBT4G6EMCARVGEMPF2CLGHMZ4X6FAA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.740734321 (12h3m30.330079308s) [noise=true , address=104.36.82.4:4802 , piece_id=IUDX2JVADG6CRXQXEBWVJBZKOXMQGYP5OA62QMPA2UGMGJS2H76Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741156128 (12h3m30.330501116s) [noise=true , address=104.36.82.10:4079 , piece_id=GGGVNJIW5LV3XLXXVD3BUI2V6ZMIBBBCNSRKUWEL525SDEK6NZ4Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741202056 (12h3m30.330547043s) [noise=true , address=45.91.116.5:4299 , piece_id=SJALZPMLAAE47SRYQJQVCKCIF5PJK6EQBIY2ZFDOWNQIYU2GIIFA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741514895 (12h3m30.330859882s) [noise=true , address=77.90.31.14:4662 , piece_id=S7CB4FTSM4X43RFRTE5AGA2MEYU47CQLDAFC6LPYYLQR5KUURRZQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741769552 (12h3m30.331114539s) [noise=true , address=45.91.116.13:4153 , piece_id=5TQO3JLJ44QSPSKKGMCLQGW2XU7ROEUGSOS2FRVAZMD3PVG4OGRA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742036122 (12h3m30.331381099s) [noise=true , address=104.36.82.2:4730 , piece_id=CYRP4OE4V4HABC5435BPXU6ZV6GRFBPCHE5GHTUPSQ7OOZ4OKNCQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742272134 (12h3m30.331617121s) [noise=true , address=104.36.82.8:4972 , piece_id=GIS3IKFGRUXF2OOJQAQRJEXIZPR7R7FOAYERHMKAHZGWOIRBI4TA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742498006 (12h3m30.331842983s) [noise=true , address=77.90.31.18:4806 , piece_id=SXF5F2YTBPCXIHAIJIABEEYYNAHFMILZG3OHE75DDLYDYUOLGE5A, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742687679 (12h3m30.332032666s) [noise=true , address=104.36.82.4:4803 , piece_id=MCGA5KUYMJPV3FY64PIXGBXU7X56EL6FFPBGNEPUBQYP6Q5YSNKA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742874586 (12h3m30.332219573s) [noise=true , address=77.90.31.21:4877 , piece_id=ATY45PDTCDCEZVN25KFBSUCZ3VGB7T3RWYIKI7JSEJKB6NTHZ6BA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743056164 (12h3m30.332401151s) [noise=true , address=104.36.82.11:4115 , piece_id=EHWHX4XIJSKXRS42RVSS6KTDIS7UEX3VYD4ZWK7LWDKPJHH5LEKA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743241057 (12h3m30.332586044s) [noise=true , address=104.36.82.13:4187 , piece_id=JJB27BC4B3ZZW74G53J22Z4GHBTIUSY5O4Z4CCQ6RKXJODFVFRUQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743428506 (12h3m30.332773493s) [noise=true , address=77.90.31.17:4771 , piece_id=3YZAG6773WIVAVGQYRKNFAKCMFFN6UXFLM7WIMJJZE24KBSDTPYA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743603631 (12h3m30.332948618s) [noise=true , address=104.36.82.12:4149 , piece_id=BXS6OKBPWYUHDNQNTS4NJDTKBVLUGDZLWRAK6JYAFDOISDD43RZQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743799917 (12h3m30.333144904s) [noise=true , address=104.36.82.9:4006 , piece_id=LULAOEC7QVMWWTQSEZONPXB4QLG5C7N4LQ3KTSZBFSKL6F5LAV4Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:32.486950039 (12h3m32.076295026s) [upload=12533]: Next returned nil segment
Aug 7 21:43:32.486967473 (12h3m32.07631246s) [upload=12533]: Waiting for error group managing segments...
Aug 7 21:43:32.783075876 (12h3m32.372420863s) [noise=true , address=104.36.82.4:4803 , piece_id=MCGA5KUYMJPV3FY64PIXGBXU7X56EL6FFPBGNEPUBQYP6Q5YSNKA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.788955976 (12h3m32.378300963s) [noise=true , address=104.36.82.4:4802 , piece_id=IUDX2JVADG6CRXQXEBWVJBZKOXMQGYP5OA62QMPA2UGMGJS2H76Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.820748373 (12h3m32.41009336s) [noise=true , address=104.36.82.12:4149 , piece_id=BXS6OKBPWYUHDNQNTS4NJDTKBVLUGDZLWRAK6JYAFDOISDD43RZQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.883252039 (12h3m32.472597026s) [noise=true , address=104.36.82.10:4079 , piece_id=GGGVNJIW5LV3XLXXVD3BUI2V6ZMIBBBCNSRKUWEL525SDEK6NZ4Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.890387611 (12h3m32.479732598s) [noise=true , address=104.36.82.13:4187 , piece_id=JJB27BC4B3ZZW74G53J22Z4GHBTIUSY5O4Z4CCQ6RKXJODFVFRUQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.895343072 (12h3m32.484688059s) [noise=true , address=104.36.82.8:4972 , piece_id=GIS3IKFGRUXF2OOJQAQRJEXIZPR7R7FOAYERHMKAHZGWOIRBI4TA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.931444153 (12h3m32.52078914s) [noise=true , address=45.91.116.5:4299 , piece_id=SJALZPMLAAE47SRYQJQVCKCIF5PJK6EQBIY2ZFDOWNQIYU2GIIFA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.037991647 (12h3m32.627336634s) [noise=true , address=104.36.82.9:4006 , piece_id=LULAOEC7QVMWWTQSEZONPXB4QLG5C7N4LQ3KTSZBFSKL6F5LAV4Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.096201326 (12h3m32.685546303s) [noise=true , address=104.36.82.11:4115 , piece_id=EHWHX4XIJSKXRS42RVSS6KTDIS7UEX3VYD4ZWK7LWDKPJHH5LEKA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.098420492 (12h3m32.68776548s) [noise=true , address=77.90.31.21:4877 , piece_id=ATY45PDTCDCEZVN25KFBSUCZ3VGB7T3RWYIKI7JSEJKB6NTHZ6BA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.110157609 (12h3m32.699502596s) [noise=true , address=77.90.31.17:4771 , piece_id=3YZAG6773WIVAVGQYRKNFAKCMFFN6UXFLM7WIMJJZE24KBSDTPYA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.270704698 (12h3m32.860049685s) [noise=true , address=45.91.116.11:4119 , piece_id=MEWVLFIXGSXTBBYJEYOJQVQBT4G6EMCARVGEMPF2CLGHMZ4X6FAA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.396150989 (12h3m32.985495966s) [noise=true , address=77.90.31.14:4662 , piece_id=S7CB4FTSM4X43RFRTE5AGA2MEYU47CQLDAFC6LPYYLQR5KUURRZQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.396176408 (12h3m32.985521395s) [seg_pos={0 0}, upload=12533]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:43:33.396417650 (12h3m32.985762637s) [noise=true , address=104.36.82.2:4730 , piece_id=CYRP4OE4V4HABC5435BPXU6ZV6GRFBPCHE5GHTUPSQ7OOZ4OKNCQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:33.396444150 (12h3m32.985789138s) [noise=true , address=77.90.31.18:4806 , piece_id=SXF5F2YTBPCXIHAIJIABEEYYNAHFMILZG3OHE75DDLYDYUOLGE5A, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:53342->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:43:33.396497823 (12h3m32.98584282s) [noise=true , address=45.91.116.13:4153 , piece_id=5TQO3JLJ44QSPSKKGMCLQGW2XU7ROEUGSOS2FRVAZMD3PVG4OGRA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:33.396510136 (12h3m32.985855123s) [seg_pos={0 0}, upload=12533]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:43:33.396576803 (12h3m32.98592179s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:43:33.396580902 (12h3m32.985925889s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:43:37.323497252 (12h3m36.912842229s) [upload=12533]: Done uploading segments...
Aug 7 21:43:37.323504276 (12h3m36.912849263s) [upload=12533]: Upload finished. err: <nil>
Aug 7 21:44:00.854272388 (12h4m0.443617375s) [upload=12543]: Starting upload
Aug 7 21:44:00.854329437 (12h4m0.443674424s) [upload=12543]: Done starting upload
Aug 7 21:44:00.854426172 (12h4m0.443771149s) [upload=12543]: Uploading segments...
Aug 7 21:44:00.854652084 (12h4m0.443997071s) [upload=12543]: Got next segment. Inline: false
Aug 7 21:44:00.854711599 (12h4m0.444056596s) [upload=12543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:44:00.854718902 (12h4m0.444063929s) [upload=12543]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:44:30.205942381 (12h4m29.795287368s) [upload=12543]: Next segment err: context deadline exceeded
Aug 7 21:44:30.206048483 (12h4m29.79539346s) [upload=12543]: Done uploading segments...
Aug 7 21:44:30.206064114 (12h4m29.795409101s) [upload=12543]: Upload finished. err: context deadline exceeded
Duration: 7.852147652s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [................................X....X.X...X................] duration:
729.014826ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:44:46 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 729.04367ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [.........................................XX.XX..X.XX........] duration:
229.260175ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 21:44:42 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 229.281696ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX....] duration:
6.032527571s
FAIL (2025-08-07 21:44:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.03254862s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [...................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X] duration:
30.001179173s
FAIL (2025-08-07 21:44:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
21:43:27.129330933 (12h3m26.71867592s) [upload=12525]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:43:27.998435988 (12h3m27.587780976s) [upload=12533]: Starting upload
Aug 7 21:43:27.998485233 (12h3m27.58783022s) [upload=12533]: Done starting upload
Aug 7 21:43:27.998649527 (12h3m27.587994504s) [upload=12533]: Uploading segments...
Aug 7 21:43:27.998680697 (12h3m27.588025674s) [upload=12533]: Got next segment. Inline: false
Aug 7 21:43:27.998702498 (12h3m27.588047475s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:43:27.998706827 (12h3m27.588051804s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:43:30.740636996 (12h3m30.329981983s) [seg_pos={0 0}, upload=12533]: Begin upload segment...
Aug 7 21:43:30.740675639 (12h3m30.330020626s) [seg_pos={0 0}, upload=12533]: Done begin upload segment.
Aug 7 21:43:30.740696189 (12h3m30.330041176s) [noise=true , address=45.91.116.11:4119 , piece_id=MEWVLFIXGSXTBBYJEYOJQVQBT4G6EMCARVGEMPF2CLGHMZ4X6FAA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.740734321 (12h3m30.330079308s) [noise=true , address=104.36.82.4:4802 , piece_id=IUDX2JVADG6CRXQXEBWVJBZKOXMQGYP5OA62QMPA2UGMGJS2H76Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741156128 (12h3m30.330501116s) [noise=true , address=104.36.82.10:4079 , piece_id=GGGVNJIW5LV3XLXXVD3BUI2V6ZMIBBBCNSRKUWEL525SDEK6NZ4Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741202056 (12h3m30.330547043s) [noise=true , address=45.91.116.5:4299 , piece_id=SJALZPMLAAE47SRYQJQVCKCIF5PJK6EQBIY2ZFDOWNQIYU2GIIFA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741514895 (12h3m30.330859882s) [noise=true , address=77.90.31.14:4662 , piece_id=S7CB4FTSM4X43RFRTE5AGA2MEYU47CQLDAFC6LPYYLQR5KUURRZQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.741769552 (12h3m30.331114539s) [noise=true , address=45.91.116.13:4153 , piece_id=5TQO3JLJ44QSPSKKGMCLQGW2XU7ROEUGSOS2FRVAZMD3PVG4OGRA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742036122 (12h3m30.331381099s) [noise=true , address=104.36.82.2:4730 , piece_id=CYRP4OE4V4HABC5435BPXU6ZV6GRFBPCHE5GHTUPSQ7OOZ4OKNCQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742272134 (12h3m30.331617121s) [noise=true , address=104.36.82.8:4972 , piece_id=GIS3IKFGRUXF2OOJQAQRJEXIZPR7R7FOAYERHMKAHZGWOIRBI4TA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742498006 (12h3m30.331842983s) [noise=true , address=77.90.31.18:4806 , piece_id=SXF5F2YTBPCXIHAIJIABEEYYNAHFMILZG3OHE75DDLYDYUOLGE5A, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742687679 (12h3m30.332032666s) [noise=true , address=104.36.82.4:4803 , piece_id=MCGA5KUYMJPV3FY64PIXGBXU7X56EL6FFPBGNEPUBQYP6Q5YSNKA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.742874586 (12h3m30.332219573s) [noise=true , address=77.90.31.21:4877 , piece_id=ATY45PDTCDCEZVN25KFBSUCZ3VGB7T3RWYIKI7JSEJKB6NTHZ6BA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743056164 (12h3m30.332401151s) [noise=true , address=104.36.82.11:4115 , piece_id=EHWHX4XIJSKXRS42RVSS6KTDIS7UEX3VYD4ZWK7LWDKPJHH5LEKA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743241057 (12h3m30.332586044s) [noise=true , address=104.36.82.13:4187 , piece_id=JJB27BC4B3ZZW74G53J22Z4GHBTIUSY5O4Z4CCQ6RKXJODFVFRUQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743428506 (12h3m30.332773493s) [noise=true , address=77.90.31.17:4771 , piece_id=3YZAG6773WIVAVGQYRKNFAKCMFFN6UXFLM7WIMJJZE24KBSDTPYA, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743603631 (12h3m30.332948618s) [noise=true , address=104.36.82.12:4149 , piece_id=BXS6OKBPWYUHDNQNTS4NJDTKBVLUGDZLWRAK6JYAFDOISDD43RZQ, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:30.743799917 (12h3m30.333144904s) [noise=true , address=104.36.82.9:4006 , piece_id=LULAOEC7QVMWWTQSEZONPXB4QLG5C7N4LQ3KTSZBFSKL6F5LAV4Q, seg_pos={0 0}, upload=12533]: Uploading piece...
Aug 7 21:43:32.486950039 (12h3m32.076295026s) [upload=12533]: Next returned nil segment
Aug 7 21:43:32.486967473 (12h3m32.07631246s) [upload=12533]: Waiting for error group managing segments...
Aug 7 21:43:32.783075876 (12h3m32.372420863s) [noise=true , address=104.36.82.4:4803 , piece_id=MCGA5KUYMJPV3FY64PIXGBXU7X56EL6FFPBGNEPUBQYP6Q5YSNKA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.788955976 (12h3m32.378300963s) [noise=true , address=104.36.82.4:4802 , piece_id=IUDX2JVADG6CRXQXEBWVJBZKOXMQGYP5OA62QMPA2UGMGJS2H76Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.820748373 (12h3m32.41009336s) [noise=true , address=104.36.82.12:4149 , piece_id=BXS6OKBPWYUHDNQNTS4NJDTKBVLUGDZLWRAK6JYAFDOISDD43RZQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.883252039 (12h3m32.472597026s) [noise=true , address=104.36.82.10:4079 , piece_id=GGGVNJIW5LV3XLXXVD3BUI2V6ZMIBBBCNSRKUWEL525SDEK6NZ4Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.890387611 (12h3m32.479732598s) [noise=true , address=104.36.82.13:4187 , piece_id=JJB27BC4B3ZZW74G53J22Z4GHBTIUSY5O4Z4CCQ6RKXJODFVFRUQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.895343072 (12h3m32.484688059s) [noise=true , address=104.36.82.8:4972 , piece_id=GIS3IKFGRUXF2OOJQAQRJEXIZPR7R7FOAYERHMKAHZGWOIRBI4TA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:32.931444153 (12h3m32.52078914s) [noise=true , address=45.91.116.5:4299 , piece_id=SJALZPMLAAE47SRYQJQVCKCIF5PJK6EQBIY2ZFDOWNQIYU2GIIFA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.037991647 (12h3m32.627336634s) [noise=true , address=104.36.82.9:4006 , piece_id=LULAOEC7QVMWWTQSEZONPXB4QLG5C7N4LQ3KTSZBFSKL6F5LAV4Q, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.096201326 (12h3m32.685546303s) [noise=true , address=104.36.82.11:4115 , piece_id=EHWHX4XIJSKXRS42RVSS6KTDIS7UEX3VYD4ZWK7LWDKPJHH5LEKA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.098420492 (12h3m32.68776548s) [noise=true , address=77.90.31.21:4877 , piece_id=ATY45PDTCDCEZVN25KFBSUCZ3VGB7T3RWYIKI7JSEJKB6NTHZ6BA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.110157609 (12h3m32.699502596s) [noise=true , address=77.90.31.17:4771 , piece_id=3YZAG6773WIVAVGQYRKNFAKCMFFN6UXFLM7WIMJJZE24KBSDTPYA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.270704698 (12h3m32.860049685s) [noise=true , address=45.91.116.11:4119 , piece_id=MEWVLFIXGSXTBBYJEYOJQVQBT4G6EMCARVGEMPF2CLGHMZ4X6FAA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.396150989 (12h3m32.985495966s) [noise=true , address=77.90.31.14:4662 , piece_id=S7CB4FTSM4X43RFRTE5AGA2MEYU47CQLDAFC6LPYYLQR5KUURRZQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: <nil>
Aug 7 21:43:33.396176408 (12h3m32.985521395s) [seg_pos={0 0}, upload=12533]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:43:33.396417650 (12h3m32.985762637s) [noise=true , address=104.36.82.2:4730 , piece_id=CYRP4OE4V4HABC5435BPXU6ZV6GRFBPCHE5GHTUPSQ7OOZ4OKNCQ, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:33.396444150 (12h3m32.985789138s) [noise=true , address=77.90.31.18:4806 , piece_id=SXF5F2YTBPCXIHAIJIABEEYYNAHFMILZG3OHE75DDLYDYUOLGE5A, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:53342->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:43:33.396497823 (12h3m32.98584282s) [noise=true , address=45.91.116.13:4153 , piece_id=5TQO3JLJ44QSPSKKGMCLQGW2XU7ROEUGSOS2FRVAZMD3PVG4OGRA, seg_pos={0 0}, upload=12533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:33.396510136 (12h3m32.985855123s) [seg_pos={0 0}, upload=12533]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:43:33.396576803 (12h3m32.98592179s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:43:33.396580902 (12h3m32.985925889s) [upload=12533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:43:37.323497252 (12h3m36.912842229s) [upload=12533]: Done uploading segments...
Aug 7 21:43:37.323504276 (12h3m36.912849263s) [upload=12533]: Upload finished. err: <nil>
Aug 7 21:44:00.854272388 (12h4m0.443617375s) [upload=12543]: Starting upload
Aug 7 21:44:00.854329437 (12h4m0.443674424s) [upload=12543]: Done starting upload
Aug 7 21:44:00.854426172 (12h4m0.443771149s) [upload=12543]: Uploading segments...
Aug 7 21:44:00.854652084 (12h4m0.443997071s) [upload=12543]: Got next segment. Inline: false
Aug 7 21:44:00.854711599 (12h4m0.444056596s) [upload=12543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:44:00.854718902 (12h4m0.444063929s) [upload=12543]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:44:30.205942381 (12h4m29.795287368s) [upload=12543]: Next segment err: context deadline exceeded
Aug 7 21:44:30.206048483 (12h4m29.79539346s) [upload=12543]: Done uploading segments...
Aug 7 21:44:30.206064114 (12h4m29.795409101s) [upload=12543]: Upload finished. err: context deadline exceeded
Duration: 30.001199421s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
36 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X....] duration:
2.554898485s
FAIL (2025-08-07 21:44:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.554919496s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
33 out of 60 is failed (threshold=3) [....................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X..] duration:
1.892238475s
FAIL (2025-08-07 21:44:23 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 1.892260608s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............................X...............X..........X....] duration:
1.858073869s
FAIL (2025-08-07 21:44:22 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil>
Duration: 1.85810045s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
27 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXXXXXXXXXXXX...X.] duration:
6.658166077s
FAIL (2025-08-07 21:44:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.658180704s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [................X.........X......X..X.............X..X......] duration:
7.084087507s
FAIL (2025-08-07 21:44:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
ad=12478]: Done starting upload
Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments...
Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false
Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment...
Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment.
Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment
Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments...
Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments...
Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil>
Duration: 7.084119088s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
27 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXXXXXXXXXXXX..X] duration:
30.000447029s
FAIL (2025-08-07 21:44:19 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.000466436s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
27 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXXXXXXXXXXXX...X] duration:
1m0.00236012s
FAIL (2025-08-07 21:44:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.002379927s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...] duration:
5.971169951s
FAIL (2025-08-07 21:44:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.971197565s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
30 out of 60 is failed (threshold=3) [............................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.X] duration:
1m0.000804001s
FAIL (2025-08-07 21:44:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== pload=12385]: Done starting upload Aug 7 21:34:31.146698838 (11h54m30.736043825s) [upload=12385]: Uploading segments... Aug 7 21:34:31.146741720 (11h54m30.736086697s) [upload=12385]: Got next segment. Inline: true Aug 7 21:34:31.146745548 (11h54m30.736090535s) [upload=12385]: Waiting for error group managing segments... Aug 7 21:34:31.146776246 (11h54m30.736121233s) [upload=12385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:31.146779913 (11h54m30.73612489s) [upload=12385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:31.146782388 (11h54m30.736127375s) [upload=12385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:30.500521706 (11h55m30.089866693s) [upload=12385]: Done uploading segments... Aug 7 21:35:30.500542195 (11h55m30.089887182s) [upload=12385]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:31.371815861 (11h55m30.961160848s) [upload=12402]: Starting upload Aug 7 21:35:31.371969656 (11h55m30.961314643s) [upload=12402]: Done starting upload Aug 7 21:35:31.372062443 (11h55m30.96140743s) [upload=12402]: Uploading segments... Aug 7 21:35:31.372108090 (11h55m30.961453077s) [upload=12402]: Got next segment. Inline: true Aug 7 21:35:31.372114412 (11h55m30.961459399s) [upload=12402]: Waiting for error group managing segments... Aug 7 21:35:31.372166783 (11h55m30.96151177s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:31.372172002 (11h55m30.961516999s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:31.372174898 (11h55m30.961519885s) [upload=12402]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:30.721260917 (11h56m30.310605904s) [upload=12402]: Done uploading segments... Aug 7 21:36:30.721280625 (11h56m30.310625612s) [upload=12402]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:33.253873027 (11h56m32.843218014s) [upload=12416]: Starting upload Aug 7 21:36:33.253941188 (11h56m32.843286175s) [upload=12416]: Done starting upload Aug 7 21:36:33.254004108 (11h56m32.843349085s) [upload=12416]: Uploading segments... Aug 7 21:36:33.254039966 (11h56m32.843384953s) [upload=12416]: Got next segment. Inline: true Aug 7 21:36:33.254043462 (11h56m32.84338845s) [upload=12416]: Waiting for error group managing segments... Aug 7 21:36:33.254074321 (11h56m32.843419308s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:33.254078290 (11h56m32.843423277s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:33.254080714 (11h56m32.843425701s) [upload=12416]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:32.603532648 (11h57m32.192877635s) [upload=12416]: Done uploading segments... Aug 7 21:37:32.603556113 (11h57m32.1929011s) [upload=12416]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:33.470049892 (11h57m33.059394869s) [upload=12434]: Starting upload Aug 7 21:37:33.470104877 (11h57m33.059449865s) [upload=12434]: Done starting upload Aug 7 21:37:33.470145565 (11h57m33.059490552s) [upload=12434]: Uploading segments... Aug 7 21:37:33.470187375 (11h57m33.059532352s) [upload=12434]: Got next segment. Inline: true Aug 7 21:37:33.470190872 (11h57m33.059535859s) [upload=12434]: Waiting for error group managing segments... Aug 7 21:37:33.470223234 (11h57m33.059568221s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:33.470226710 (11h57m33.059571697s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:33.470228904 (11h57m33.059573881s) [upload=12434]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:32.827667037 (11h58m32.417012024s) [upload=12434]: Done uploading segments... Aug 7 21:38:32.827687857 (11h58m32.417032834s) [upload=12434]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:33.703666664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.00082442s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
33 out of 60 is failed (threshold=3) [.....................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X.] duration:
2.238278112s
FAIL (2025-08-07 21:44:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.238298743s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [..........................................XX.XX..X.XX.......] duration:
288.653389ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 21:44:05 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 288.678907ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
33 out of 60 is failed (threshold=3) [......................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X] duration:
30.00083218s
FAIL (2025-08-07 21:44:03 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.000854083s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [.................................X....X.X...X...............] duration:
793.78616ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:44:00 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 793.807793ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
36 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...] duration:
2.344093075s
FAIL (2025-08-07 21:43:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.344125566s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..] duration:
6.059253217s
FAIL (2025-08-07 21:43:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.059274799s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
25 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXXXXXXXXXXX.....] duration:
6.175859035s
FAIL (2025-08-07 21:43:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
2975 (11h51m26.263997962s) [noise=true , address=104.36.82.45:4937 , piece_id=EBUAJKXBK6RZK7T46KXJT5P5LINBFVZCEPGVGG5OZ66SYNOA2RSQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: <nil>
Aug 7 21:31:26.680556480 (11h51m26.269901467s) [noise=true , address=45.91.116.10:4110 , piece_id=F4ZMNLCCFQCBFB6FOPOJFTGPIL5BFYT6TPGL5C7SVLIHMTY4EJJA, seg_pos={0 0}, upload=12321]: Done uploading piece. err: <nil>
Aug 7 21:31:26.680580406 (11h51m26.269925393s) [seg_pos={0 0}, upload=12321]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:31:26.680752626 (11h51m26.270097613s) [noise=true , address=45.91.116.6:4983 , piece_id=ZWH64KJWCVSTEGVN25WW35HRHTAO3BE3SPKKALNRT7NNLUKB4L4Q, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122trJbDHuqeBMQHZT97HP2PgkeuLyUiDiuRm4nayu6ZTH7tat6): protocol: expected piece hash; context canceled; EOF
Aug 7 21:31:26.680804515 (11h51m26.270149492s) [noise=true , address=45.91.116.7:4037 , piece_id=WY2F64PFXEGFVE4M6PUHHUYIMTI2KB4CB73DQADZK4QGIEZ2YCYQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK): protocol: expected piece hash; context canceled; EOF
Aug 7 21:31:26.680954802 (11h51m26.270299789s) [noise=true , address=104.36.82.61:4375 , piece_id=5LQOQ3MR3EA36DIYLOVO6YDKF6EP2HPHMPG4P6MN76A4IA5BTBEQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: write tcp 188.166.225.170:45604->104.36.82.61:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:31:26.680994198 (11h51m26.270339185s) [noise=true , address=104.36.82.8:4977 , piece_id=VY47HC4FF6JU4QQTDVZ62NOIEGWSROTWYMGCB7XXNHJ3GZNJATMQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT): protocol: write tcp 188.166.225.170:36156->104.36.82.8:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:31:26.681016300 (11h51m26.270361287s) [seg_pos={0 0}, upload=12321]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:31:26.681067097 (11h51m26.270412074s) [upload=12321]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:31:26.681072137 (11h51m26.270417114s) [upload=12321]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:32:39.816785277 (11h52m39.406130264s) [upload=12321]: Done uploading segments...
Aug 7 21:32:39.816815415 (11h52m39.406160402s) [upload=12321]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:32:40.684664778 (11h52m40.274009755s) [upload=12355]: Starting upload
Aug 7 21:32:40.684721386 (11h52m40.274066373s) [upload=12355]: Done starting upload
Aug 7 21:32:40.684776000 (11h52m40.274121017s) [upload=12355]: Uploading segments...
Aug 7 21:32:40.684893314 (11h52m40.274238292s) [upload=12355]: Got next segment. Inline: false
Aug 7 21:32:40.684916699 (11h52m40.274261686s) [upload=12355]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:32:40.684920466 (11h52m40.274265443s) [upload=12355]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:33:40.036921304 (11h53m39.626266281s) [upload=12355]: Next segment err: context deadline exceeded
Aug 7 21:33:40.037128099 (11h53m39.626473086s) [upload=12355]: Done uploading segments...
Aug 7 21:33:40.037148919 (11h53m39.626493896s) [upload=12355]: Upload finished. err: context deadline exceeded
Aug 7 21:33:40.899438535 (11h53m40.488783522s) [upload=12372]: Starting upload
Aug 7 21:33:40.899490825 (11h53m40.488835832s) [upload=12372]: Done starting upload
Aug 7 21:33:40.899590144 (11h53m40.488935131s) [upload=12372]: Uploading segments...
Aug 7 21:33:40.899771832 (11h53m40.489116819s) [upload=12372]: Got next segment. Inline: false
Aug 7 21:33:40.899795227 (11h53m40.489140214s) [upload=12372]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:33:40.899799275 (11h53m40.489144252s) [upload=12372]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:34:40.257761531 (11h54m39.847106518s) [upload=12372]: Next segment err: context deadline exceeded
Aug 7 21:34:40.257885859 (11h54m39.847230856s) [upload=12372]: Done uploading segments...
Aug 7 21:34:40.257904695 (11h54m39.847249672s) [upload=12372]: Upload finished. err: context deadline exceeded
Aug 7 21:34:41.144124896 (11h54m40.733469873s) [upload=12386]: Starting upload
Aug 7 21:34:41.144233103 (11h54m40.73357813s) [upload=12386]: Done starting upload
Aug 7 21:34:41.144343113 (11h54m40.733688101s) [upload=12386]: Uploading segments...
Aug 7 21:34:41.144700337 (11h54m40.734045364s) [upload=12386]: Got next segment. Inline: false
Aug 7 21:34:41.144760612 (11h54m40.734105639s) [upload=12386]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:34:41.144768206 (11h54m40.734113223s) [upload=12386]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:35:40.491924814 (11h55m40.081269811s) [upload=12386]: Next segment err: context deadline exceeded
Aug 7 21:35:40.492048891 (11h55m40.081393878s) [upload=12386]: Done uploading segments...
Aug 7 21:35:40.492066364 (11h55m40.081411351s) [upload=12386]: Upload finished. err: context deadline exceeded
Aug 7 21:35:41.353263670 (11h55m40.942608657s) [upload=12405]: Starting upload
Aug 7 21:35:41.353369352 (11h55m40.942714339s) [upload=12405]: Done starting upload
Aug 7 21:35:41.353452601 (11h55m40.942797588s) [upload=12405]: Uploading segments...
Aug 7 21:35:41.353579473 (11h55m40.942924511s) [upload=12405]: Got next segment. Inline: false
Aug 7 21:35:41.353621805 (11h55m40.942966792s) [upload=12405]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:35:41.353626053 (11h55m40.94297104s) [upload=12405]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:36:40.713868241 (11h56m40.303213228s) [upload=12405]: Next segment err: context deadline exceeded
Aug 7 21:36:40.713958674 (11h56m40.303303651s) [upload=12405]: Done uploading segments...
Aug 7 21:36:40.713972460 (11h56m40.303317447s) [upload=12405]: Upload finished. err: context deadline exceeded
Aug 7 21:36:41.596698707 (11h56m41.186043744s) [upload=12418]: Starting upload
Aug 7 21:36:41.596808026 (11h56m41.186153063s) [upload=12418]: Done starting upload
Aug 7 21:36:41.596885475 (11h56m41.186230462s) [upload=12418]: Uploading segments...
Aug 7 21:36:41.597098722 (11h56m41.186443709s) [upload=12418]: Got next segment. Inline: false
Aug 7 21:36:41.597141925 (11h56m41.186486912s) [upload=12418]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:36:41.597146604 (11h56m41.186491591s) [upload=12418]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:37:40.947744552 (11h57m40.537089539s) [upload=12418]: Next segment err: context deadline exceeded
Aug 7 21:37:40.947825697 (11h57m40.537170684s) [upload=12418]: Done uploading segments...
Aug 7 21:37:40.947843592 (11h57m40.537188579s) [upload=12418]: Upload finished. err: context deadline exceeded
Aug 7 21:37:41.823629920 (11h57m41.412974907s) [upload=12437]: Starting upload
Aug 7 21:37:41.823680908 (11h57m41.413025895s) [upload=12437]: Done starting upload
Aug 7 21:37:41.823826305 (11h57m41.413171293s) [upload=12437]: Uploading segments...
Aug 7 21:37:41.823857055 (11h57m41.413202052s) [upload=12437]: Got next segment. Inline: false
Aug 7 21:37:41.823914255 (11h57m41.413259242s) [upload=12437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:37:41.823918503 (11h57m41.41326348s) [upload=12437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:38:41.179029912 (11h58m40.768374899s) [upload=12437]: Next segment err: context deadline exceeded
Aug 7 21:38:41.179128490 (11h58m40.768473477s) [upload=12437]: Done uploading segments...
Aug 7 21:38:41.179142979 (11h58m40.768487966s) [upload=12437]: Upload finished. err: context deadline exceeded
Aug 7 21:38:42.074438453 (11h58m41.66378344s) [upload=12451]: Starting upload
Aug 7 21:38:42.074516963 (11h58m41.66386195s) [upload=12451]: Done starting upload
Aug 7 21:38:42.074605202 (11h58m41.663950189s) [upload=12451]: Uploading segments...
Aug 7 21:38:42.074689222 (11h58m41.664034209s) [upload=12451]: Got next segment. Inline: false
Aug 7 21:38:42.074715102 (11h58m41.664060089s) [upload=12451]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:38:42.074718829 (11h58m41.664063816s) [upload=12451]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:39:41.428394360 (11h59m41.017739347s) [upload=12451]: Next segment err: context deadline exceeded
Aug 7 21:39:41.428496375 (11h59m41.017841372s) [upload=12451]: Done uploading segments...
Aug 7 21:39:41.428519450 (11h59m41.017864437s) [upload=12451]: Upload finished. err: context deadline exceeded
Aug 7 21:39:42.313134109 (11h59m41.902479096s) [upload=12468]: Starting upload
Aug 7 21:39:42.313191258 (11h59m41.902536245s) [upload=12468]: Done starting upload
Aug 7 21:39:42.313256903 (11h59m41.90260189s) [upload=12468]: Uploading segments...
Aug 7 21:39:42.313381803 (11h59m41.90272684s) [upload=12468]: Got next segment. Inline: false
Aug 7 21:39:42.313472736 (11h59m41.902817763s) [upload=12468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:39:42.313480300 (11h59m41.902825308s) [upload=12468]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:40:41.669512466 (12h0m41.258857454s) [upload=12468]: Next segment err: context deadline exceeded
Aug 7 21:40:41.669582440 (12h0m41.258927427s) [upload=12468]: Done uploading segments...
Aug 7 21:40:41.669600465 (12h0m41.258945452s) [upload=12468]: Upload finished. err: context deadline exceeded
Duration: 6.175884813s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
36 out of 60 is failed (threshold=3) [....................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.] duration:
13.254620328s
FAIL (2025-08-07 21:43:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
anceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: read tcp 188.166.225.170:33304->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:42:42.621432934 (12h2m42.210777921s) [seg_pos={0 0}, upload=12517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:42:42.621510242 (12h2m42.210855229s) [upload=12517]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:42:42.621514751 (12h2m42.210859738s) [upload=12517]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:42:43.408155141 (12h2m42.997500128s) [upload=12517]: Done uploading segments...
Aug 7 21:42:43.408174438 (12h2m42.997519425s) [upload=12517]: Upload finished. err: <nil>
Aug 7 21:42:54.608286337 (12h2m54.197631324s) [upload=12525]: Starting upload
Aug 7 21:42:54.608338167 (12h2m54.197683144s) [upload=12525]: Done starting upload
Aug 7 21:42:54.608520635 (12h2m54.197865622s) [upload=12525]: Uploading segments...
Aug 7 21:42:54.608546946 (12h2m54.197891933s) [upload=12525]: Got next segment. Inline: false
Aug 7 21:42:54.608567746 (12h2m54.197912733s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:42:54.608573056 (12h2m54.197918043s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:43:19.700246037 (12h3m19.289591024s) [seg_pos={0 0}, upload=12525]: Begin upload segment...
Aug 7 21:43:19.700314288 (12h3m19.289659275s) [seg_pos={0 0}, upload=12525]: Done begin upload segment.
Aug 7 21:43:19.700365777 (12h3m19.289710784s) [noise=true , address=45.91.116.4:4938 , piece_id=TRRKGOSDIZENSDSZVGCIDIVFDJ2EC33O4PDB6KWZ7JGTD7ZTOLXQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.700814365 (12h3m19.290159352s) [noise=true , address=77.90.31.16:4735 , piece_id=IK3WGGZ2YHGYGWFHK3CNQLQGX3HRDHWULRJRIOPVHOOBTLLEUW4A, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701273834 (12h3m19.290618851s) [noise=true , address=104.36.82.12:4150 , piece_id=RC45TEYSPLOSI5B6AMSRZO64SGSNFL7VQKV4NYIOEAVVTFOOI2LQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701325392 (12h3m19.290670379s) [noise=true , address=45.91.116.12:4227 , piece_id=VAXOIPMDD2WACHN4OYTSTDZRTUBCNCU6ID2QX7QRESU6ZI4OH2NQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701635176 (12h3m19.290980163s) [noise=true , address=104.36.82.5:4839 , piece_id=NQLAXGPOJ7JD5BKHGCH4TJLAW6OJRHSBDMQYQOBMF6HSGJO2BWBQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701964576 (12h3m19.291309563s) [noise=true , address=77.90.31.19:4842 , piece_id=TH63O3O3N77LNJWDKLPGFF67CEOK4UMFLZZI3K45JG4P6YHXI66A, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.702264640 (12h3m19.291609617s) [noise=true , address=77.90.31.14:4662 , piece_id=7NJKDTMMGJXLWN6UX74WLLHN42HAU67EJQVLPHJFIOQZMMUNNMTA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.703999258 (12h3m19.293344255s) [noise=true , address=77.90.31.21:4877 , piece_id=355KVO3SAR4XGRIKYIMB7D5TGWFURDDTG3EYQ4QPGLNIQOAASZZA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.704447376 (12h3m19.293792363s) [noise=true , address=77.90.31.12:4589 , piece_id=LT5J4PH6F7ITQ33L76RMM2XOVXK5JPFECHKSNJDV3AFMQKRTTWYQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.704822112 (12h3m19.294167099s) [noise=true , address=77.90.31.18:4805 , piece_id=YP6TZCVAIOSLWDCZBIVJVULVQ4R5O6BKO6D2WW44M3PWWZBMQVRQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705200266 (12h3m19.294545253s) [noise=true , address=45.91.116.10:4082 , piece_id=ISRKZRQGQYX62S4UTVIGCOPWFHXXDT3EP6SK3UWVJMTZ554TSETQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705538403 (12h3m19.29488339s) [noise=true , address=45.91.116.13:4155 , piece_id=M2CVBGERM4SP3XRRKOA7YNPVTYMGW2BOWBVGH2Q672NK5AEHZ34Q, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705925874 (12h3m19.295270861s) [noise=true , address=45.91.116.13:4153 , piece_id=CXBT3KDSMBPHSZJDJ6KBTGJLVKW3TZ5MW7V4FZ7VPVLGYCJYRWMA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706351669 (12h3m19.295696656s) [noise=true , address=104.36.82.9:4005 , piece_id=KIBKKSRCA7TJMCJGCDE4NGLV6ZKOGD3OFFJZ6ILE3TPY4XK4GG7Q, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706656893 (12h3m19.29600188s) [noise=true , address=104.36.82.6:4875 , piece_id=3CQ5C3COW4XG3KYMSVDARHZTSOHZMBZXH3D2CAICFON6CP6DW4WQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706958440 (12h3m19.296303427s) [noise=true , address=45.91.116.9:4261 , piece_id=6OWH7KTHHCMOSY2M3JTHOVKHZ7VDGMEYOJWYRO5TVJ2RYP2RUHUQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:21.450498157 (12h3m21.039843154s) [upload=12525]: Next returned nil segment
Aug 7 21:43:21.450514448 (12h3m21.039859435s) [upload=12525]: Waiting for error group managing segments...
Aug 7 21:43:21.777426888 (12h3m21.366771875s) [noise=true , address=45.91.116.13:4155 , piece_id=M2CVBGERM4SP3XRRKOA7YNPVTYMGW2BOWBVGH2Q672NK5AEHZ34Q, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:21.818986662 (12h3m21.408331639s) [noise=true , address=77.90.31.12:4589 , piece_id=LT5J4PH6F7ITQ33L76RMM2XOVXK5JPFECHKSNJDV3AFMQKRTTWYQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:21.878863543 (12h3m21.46820853s) [noise=true , address=77.90.31.19:4842 , piece_id=TH63O3O3N77LNJWDKLPGFF67CEOK4UMFLZZI3K45JG4P6YHXI66A, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.007316987 (12h3m21.596661974s) [noise=true , address=77.90.31.21:4877 , piece_id=355KVO3SAR4XGRIKYIMB7D5TGWFURDDTG3EYQ4QPGLNIQOAASZZA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.035310837 (12h3m21.624655844s) [noise=true , address=77.90.31.16:4735 , piece_id=IK3WGGZ2YHGYGWFHK3CNQLQGX3HRDHWULRJRIOPVHOOBTLLEUW4A, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.045857436 (12h3m21.635202423s) [noise=true , address=104.36.82.5:4839 , piece_id=NQLAXGPOJ7JD5BKHGCH4TJLAW6OJRHSBDMQYQOBMF6HSGJO2BWBQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.052924446 (12h3m21.642269433s) [noise=true , address=104.36.82.12:4150 , piece_id=RC45TEYSPLOSI5B6AMSRZO64SGSNFL7VQKV4NYIOEAVVTFOOI2LQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.115327661 (12h3m21.704672648s) [noise=true , address=45.91.116.10:4082 , piece_id=ISRKZRQGQYX62S4UTVIGCOPWFHXXDT3EP6SK3UWVJMTZ554TSETQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.129092156 (12h3m21.718437143s) [noise=true , address=77.90.31.14:4662 , piece_id=7NJKDTMMGJXLWN6UX74WLLHN42HAU67EJQVLPHJFIOQZMMUNNMTA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.174395092 (12h3m21.763740079s) [noise=true , address=45.91.116.13:4153 , piece_id=CXBT3KDSMBPHSZJDJ6KBTGJLVKW3TZ5MW7V4FZ7VPVLGYCJYRWMA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.192234153 (12h3m21.78157914s) [noise=true , address=45.91.116.12:4227 , piece_id=VAXOIPMDD2WACHN4OYTSTDZRTUBCNCU6ID2QX7QRESU6ZI4OH2NQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.224620877 (12h3m21.813965864s) [noise=true , address=45.91.116.4:4938 , piece_id=TRRKGOSDIZENSDSZVGCIDIVFDJ2EC33O4PDB6KWZ7JGTD7ZTOLXQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.230230970 (12h3m21.819575957s) [noise=true , address=45.91.116.9:4261 , piece_id=6OWH7KTHHCMOSY2M3JTHOVKHZ7VDGMEYOJWYRO5TVJ2RYP2RUHUQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.230257020 (12h3m21.819601997s) [seg_pos={0 0}, upload=12525]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:43:22.230389424 (12h3m21.819734401s) [noise=true , address=104.36.82.9:4005 , piece_id=KIBKKSRCA7TJMCJGCDE4NGLV6ZKOGD3OFFJZ6ILE3TPY4XK4GG7Q, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:22.230489015 (12h3m21.819834002s) [noise=true , address=77.90.31.18:4805 , piece_id=YP6TZCVAIOSLWDCZBIVJVULVQ4R5O6BKO6D2WW44M3PWWZBMQVRQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled
Aug 7 21:43:22.230573256 (12h3m21.819918243s) [noise=true , address=104.36.82.6:4875 , piece_id=3CQ5C3COW4XG3KYMSVDARHZTSOHZMBZXH3D2CAICFON6CP6DW4WQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection; write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection
Aug 7 21:43:22.230589486 (12h3m21.819934473s) [seg_pos={0 0}, upload=12525]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:43:22.230647637 (12h3m21.819992625s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:43:22.230651344 (12h3m21.819996331s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:43:27.129306256 (12h3m26.718651243s) [upload=12525]: Done uploading segments...
Aug 7 21:43:27.129330933 (12h3m26.71867592s) [upload=12525]: Upload finished. err: metaclient: context deadline exceeded
Duration: 13.254639244s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.................X.........X......X..X.............X..X.....] duration:
6.033511218s
FAIL (2025-08-07 21:43:39 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
ad=12478]: Done starting upload
Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments...
Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false
Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment...
Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment.
Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece...
Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment
Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments...
Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil>
Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF
Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments...
Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil>
Duration: 6.033541786s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
32 out of 60 is failed (threshold=3) [........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...] duration:
1.967073206s
FAIL (2025-08-07 21:43:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 745s) [upload=12364]: Done starting upload Aug 7 21:33:17.618658725 (11h53m17.208003742s) [upload=12364]: Uploading segments... Aug 7 21:33:17.618713580 (11h53m17.208058597s) [upload=12364]: Got next segment. Inline: true Aug 7 21:33:17.618720533 (11h53m17.20806553s) [upload=12364]: Waiting for error group managing segments... Aug 7 21:33:17.618772433 (11h53m17.20811743s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:33:17.618779636 (11h53m17.208124643s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:33:17.618784776 (11h53m17.208129793s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:33:46.957323530 (11h53m46.546668577s) [upload=12364]: Done uploading segments... Aug 7 21:33:46.957345773 (11h53m46.54669075s) [upload=12364]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:34:19.197336641 (11h54m18.786681628s) [upload=12382]: Starting upload Aug 7 21:34:19.197461931 (11h54m18.786806918s) [upload=12382]: Done starting upload Aug 7 21:34:19.197512228 (11h54m18.786857225s) [upload=12382]: Uploading segments... Aug 7 21:34:19.197561282 (11h54m18.786906269s) [upload=12382]: Got next segment. Inline: true Aug 7 21:34:19.197565189 (11h54m18.786910166s) [upload=12382]: Waiting for error group managing segments... Aug 7 21:34:19.197610725 (11h54m18.786955723s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:19.197615966 (11h54m18.786960953s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:19.197619142 (11h54m18.786964139s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.535954936 (11h54m48.125299923s) [upload=12382]: Done uploading segments... Aug 7 21:34:48.535978650 (11h54m48.125323637s) [upload=12382]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:05.991757765 (11h55m5.581102752s) [upload=12393]: Starting upload Aug 7 21:35:05.991822780 (11h55m5.581167767s) [upload=12393]: Done starting upload Aug 7 21:35:05.991946456 (11h55m5.581291443s) [upload=12393]: Uploading segments... Aug 7 21:35:05.991988116 (11h55m5.581333104s) [upload=12393]: Got next segment. Inline: true Aug 7 21:35:05.991994929 (11h55m5.581339996s) [upload=12393]: Waiting for error group managing segments... Aug 7 21:35:05.992042741 (11h55m5.581387728s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:05.992048021 (11h55m5.581393018s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:05.992051768 (11h55m5.581396755s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:35.339368736 (11h55m34.928713723s) [upload=12393]: Done uploading segments... Aug 7 21:35:35.339503534 (11h55m34.928848522s) [upload=12393]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:12.952326182 (11h56m12.541671159s) [upload=12411]: Starting upload Aug 7 21:36:12.952385075 (11h56m12.541730052s) [upload=12411]: Done starting upload Aug 7 21:36:12.952428238 (11h56m12.541773225s) [upload=12411]: Uploading segments... Aug 7 21:36:12.952455510 (11h56m12.541800497s) [upload=12411]: Got next segment. Inline: true Aug 7 21:36:12.952459417 (11h56m12.541804404s) [upload=12411]: Waiting for error group managing segments... Aug 7 21:36:12.952507650 (11h56m12.541852637s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:12.952514352 (11h56m12.541859349s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:12.952518550 (11h56m12.541863537s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:42.299158316 (11h56m41.888503304s) [upload=12411]: Done uploading segments... Aug 7 21:36:42.299175039 (11h56m41.888520026s) [upload=12411]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.967115938s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
36 out of 60 is failed (threshold=3) [.....................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X] duration:
33.172171102s
FAIL (2025-08-07 21:43:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
anceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: read tcp 188.166.225.170:33304->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:42:42.621432934 (12h2m42.210777921s) [seg_pos={0 0}, upload=12517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:42:42.621510242 (12h2m42.210855229s) [upload=12517]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:42:42.621514751 (12h2m42.210859738s) [upload=12517]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:42:43.408155141 (12h2m42.997500128s) [upload=12517]: Done uploading segments...
Aug 7 21:42:43.408174438 (12h2m42.997519425s) [upload=12517]: Upload finished. err: <nil>
Aug 7 21:42:54.608286337 (12h2m54.197631324s) [upload=12525]: Starting upload
Aug 7 21:42:54.608338167 (12h2m54.197683144s) [upload=12525]: Done starting upload
Aug 7 21:42:54.608520635 (12h2m54.197865622s) [upload=12525]: Uploading segments...
Aug 7 21:42:54.608546946 (12h2m54.197891933s) [upload=12525]: Got next segment. Inline: false
Aug 7 21:42:54.608567746 (12h2m54.197912733s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:42:54.608573056 (12h2m54.197918043s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:43:19.700246037 (12h3m19.289591024s) [seg_pos={0 0}, upload=12525]: Begin upload segment...
Aug 7 21:43:19.700314288 (12h3m19.289659275s) [seg_pos={0 0}, upload=12525]: Done begin upload segment.
Aug 7 21:43:19.700365777 (12h3m19.289710784s) [noise=true , address=45.91.116.4:4938 , piece_id=TRRKGOSDIZENSDSZVGCIDIVFDJ2EC33O4PDB6KWZ7JGTD7ZTOLXQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.700814365 (12h3m19.290159352s) [noise=true , address=77.90.31.16:4735 , piece_id=IK3WGGZ2YHGYGWFHK3CNQLQGX3HRDHWULRJRIOPVHOOBTLLEUW4A, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701273834 (12h3m19.290618851s) [noise=true , address=104.36.82.12:4150 , piece_id=RC45TEYSPLOSI5B6AMSRZO64SGSNFL7VQKV4NYIOEAVVTFOOI2LQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701325392 (12h3m19.290670379s) [noise=true , address=45.91.116.12:4227 , piece_id=VAXOIPMDD2WACHN4OYTSTDZRTUBCNCU6ID2QX7QRESU6ZI4OH2NQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701635176 (12h3m19.290980163s) [noise=true , address=104.36.82.5:4839 , piece_id=NQLAXGPOJ7JD5BKHGCH4TJLAW6OJRHSBDMQYQOBMF6HSGJO2BWBQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.701964576 (12h3m19.291309563s) [noise=true , address=77.90.31.19:4842 , piece_id=TH63O3O3N77LNJWDKLPGFF67CEOK4UMFLZZI3K45JG4P6YHXI66A, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.702264640 (12h3m19.291609617s) [noise=true , address=77.90.31.14:4662 , piece_id=7NJKDTMMGJXLWN6UX74WLLHN42HAU67EJQVLPHJFIOQZMMUNNMTA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.703999258 (12h3m19.293344255s) [noise=true , address=77.90.31.21:4877 , piece_id=355KVO3SAR4XGRIKYIMB7D5TGWFURDDTG3EYQ4QPGLNIQOAASZZA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.704447376 (12h3m19.293792363s) [noise=true , address=77.90.31.12:4589 , piece_id=LT5J4PH6F7ITQ33L76RMM2XOVXK5JPFECHKSNJDV3AFMQKRTTWYQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.704822112 (12h3m19.294167099s) [noise=true , address=77.90.31.18:4805 , piece_id=YP6TZCVAIOSLWDCZBIVJVULVQ4R5O6BKO6D2WW44M3PWWZBMQVRQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705200266 (12h3m19.294545253s) [noise=true , address=45.91.116.10:4082 , piece_id=ISRKZRQGQYX62S4UTVIGCOPWFHXXDT3EP6SK3UWVJMTZ554TSETQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705538403 (12h3m19.29488339s) [noise=true , address=45.91.116.13:4155 , piece_id=M2CVBGERM4SP3XRRKOA7YNPVTYMGW2BOWBVGH2Q672NK5AEHZ34Q, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.705925874 (12h3m19.295270861s) [noise=true , address=45.91.116.13:4153 , piece_id=CXBT3KDSMBPHSZJDJ6KBTGJLVKW3TZ5MW7V4FZ7VPVLGYCJYRWMA, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706351669 (12h3m19.295696656s) [noise=true , address=104.36.82.9:4005 , piece_id=KIBKKSRCA7TJMCJGCDE4NGLV6ZKOGD3OFFJZ6ILE3TPY4XK4GG7Q, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706656893 (12h3m19.29600188s) [noise=true , address=104.36.82.6:4875 , piece_id=3CQ5C3COW4XG3KYMSVDARHZTSOHZMBZXH3D2CAICFON6CP6DW4WQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:19.706958440 (12h3m19.296303427s) [noise=true , address=45.91.116.9:4261 , piece_id=6OWH7KTHHCMOSY2M3JTHOVKHZ7VDGMEYOJWYRO5TVJ2RYP2RUHUQ, seg_pos={0 0}, upload=12525]: Uploading piece...
Aug 7 21:43:21.450498157 (12h3m21.039843154s) [upload=12525]: Next returned nil segment
Aug 7 21:43:21.450514448 (12h3m21.039859435s) [upload=12525]: Waiting for error group managing segments...
Aug 7 21:43:21.777426888 (12h3m21.366771875s) [noise=true , address=45.91.116.13:4155 , piece_id=M2CVBGERM4SP3XRRKOA7YNPVTYMGW2BOWBVGH2Q672NK5AEHZ34Q, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:21.818986662 (12h3m21.408331639s) [noise=true , address=77.90.31.12:4589 , piece_id=LT5J4PH6F7ITQ33L76RMM2XOVXK5JPFECHKSNJDV3AFMQKRTTWYQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:21.878863543 (12h3m21.46820853s) [noise=true , address=77.90.31.19:4842 , piece_id=TH63O3O3N77LNJWDKLPGFF67CEOK4UMFLZZI3K45JG4P6YHXI66A, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.007316987 (12h3m21.596661974s) [noise=true , address=77.90.31.21:4877 , piece_id=355KVO3SAR4XGRIKYIMB7D5TGWFURDDTG3EYQ4QPGLNIQOAASZZA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.035310837 (12h3m21.624655844s) [noise=true , address=77.90.31.16:4735 , piece_id=IK3WGGZ2YHGYGWFHK3CNQLQGX3HRDHWULRJRIOPVHOOBTLLEUW4A, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.045857436 (12h3m21.635202423s) [noise=true , address=104.36.82.5:4839 , piece_id=NQLAXGPOJ7JD5BKHGCH4TJLAW6OJRHSBDMQYQOBMF6HSGJO2BWBQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.052924446 (12h3m21.642269433s) [noise=true , address=104.36.82.12:4150 , piece_id=RC45TEYSPLOSI5B6AMSRZO64SGSNFL7VQKV4NYIOEAVVTFOOI2LQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.115327661 (12h3m21.704672648s) [noise=true , address=45.91.116.10:4082 , piece_id=ISRKZRQGQYX62S4UTVIGCOPWFHXXDT3EP6SK3UWVJMTZ554TSETQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.129092156 (12h3m21.718437143s) [noise=true , address=77.90.31.14:4662 , piece_id=7NJKDTMMGJXLWN6UX74WLLHN42HAU67EJQVLPHJFIOQZMMUNNMTA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.174395092 (12h3m21.763740079s) [noise=true , address=45.91.116.13:4153 , piece_id=CXBT3KDSMBPHSZJDJ6KBTGJLVKW3TZ5MW7V4FZ7VPVLGYCJYRWMA, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.192234153 (12h3m21.78157914s) [noise=true , address=45.91.116.12:4227 , piece_id=VAXOIPMDD2WACHN4OYTSTDZRTUBCNCU6ID2QX7QRESU6ZI4OH2NQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.224620877 (12h3m21.813965864s) [noise=true , address=45.91.116.4:4938 , piece_id=TRRKGOSDIZENSDSZVGCIDIVFDJ2EC33O4PDB6KWZ7JGTD7ZTOLXQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.230230970 (12h3m21.819575957s) [noise=true , address=45.91.116.9:4261 , piece_id=6OWH7KTHHCMOSY2M3JTHOVKHZ7VDGMEYOJWYRO5TVJ2RYP2RUHUQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: <nil>
Aug 7 21:43:22.230257020 (12h3m21.819601997s) [seg_pos={0 0}, upload=12525]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:43:22.230389424 (12h3m21.819734401s) [noise=true , address=104.36.82.9:4005 , piece_id=KIBKKSRCA7TJMCJGCDE4NGLV6ZKOGD3OFFJZ6ILE3TPY4XK4GG7Q, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:43:22.230489015 (12h3m21.819834002s) [noise=true , address=77.90.31.18:4805 , piece_id=YP6TZCVAIOSLWDCZBIVJVULVQ4R5O6BKO6D2WW44M3PWWZBMQVRQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled
Aug 7 21:43:22.230573256 (12h3m21.819918243s) [noise=true , address=104.36.82.6:4875 , piece_id=3CQ5C3COW4XG3KYMSVDARHZTSOHZMBZXH3D2CAICFON6CP6DW4WQ, seg_pos={0 0}, upload=12525]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection; write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53472->104.36.82.6:4875: use of closed network connection
Aug 7 21:43:22.230589486 (12h3m21.819934473s) [seg_pos={0 0}, upload=12525]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:43:22.230647637 (12h3m21.819992625s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:43:22.230651344 (12h3m21.819996331s) [upload=12525]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:43:27.129306256 (12h3m26.718651243s) [upload=12525]: Done uploading segments...
Aug 7 21:43:27.129330933 (12h3m26.71867592s) [upload=12525]: Upload finished. err: metaclient: context deadline exceeded
Duration: 33.172210918s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
7 out of 60 is failed (threshold=3) [...........................................XX.XX..X.XX......] duration:
238.552453ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-07 21:43:22 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded
Duration: 238.566349ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............................X...............X..........X...] duration:
2.132236908s
FAIL (2025-08-07 21:43:20 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil>
Duration: 2.132261393s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
32 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..] duration:
14.858381691s
FAIL (2025-08-07 21:43:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 745s) [upload=12364]: Done starting upload Aug 7 21:33:17.618658725 (11h53m17.208003742s) [upload=12364]: Uploading segments... Aug 7 21:33:17.618713580 (11h53m17.208058597s) [upload=12364]: Got next segment. Inline: true Aug 7 21:33:17.618720533 (11h53m17.20806553s) [upload=12364]: Waiting for error group managing segments... Aug 7 21:33:17.618772433 (11h53m17.20811743s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:33:17.618779636 (11h53m17.208124643s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:33:17.618784776 (11h53m17.208129793s) [upload=12364]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:33:46.957323530 (11h53m46.546668577s) [upload=12364]: Done uploading segments... Aug 7 21:33:46.957345773 (11h53m46.54669075s) [upload=12364]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:34:19.197336641 (11h54m18.786681628s) [upload=12382]: Starting upload Aug 7 21:34:19.197461931 (11h54m18.786806918s) [upload=12382]: Done starting upload Aug 7 21:34:19.197512228 (11h54m18.786857225s) [upload=12382]: Uploading segments... Aug 7 21:34:19.197561282 (11h54m18.786906269s) [upload=12382]: Got next segment. Inline: true Aug 7 21:34:19.197565189 (11h54m18.786910166s) [upload=12382]: Waiting for error group managing segments... Aug 7 21:34:19.197610725 (11h54m18.786955723s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:34:19.197615966 (11h54m18.786960953s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:19.197619142 (11h54m18.786964139s) [upload=12382]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.535954936 (11h54m48.125299923s) [upload=12382]: Done uploading segments... Aug 7 21:34:48.535978650 (11h54m48.125323637s) [upload=12382]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:35:05.991757765 (11h55m5.581102752s) [upload=12393]: Starting upload Aug 7 21:35:05.991822780 (11h55m5.581167767s) [upload=12393]: Done starting upload Aug 7 21:35:05.991946456 (11h55m5.581291443s) [upload=12393]: Uploading segments... Aug 7 21:35:05.991988116 (11h55m5.581333104s) [upload=12393]: Got next segment. Inline: true Aug 7 21:35:05.991994929 (11h55m5.581339996s) [upload=12393]: Waiting for error group managing segments... Aug 7 21:35:05.992042741 (11h55m5.581387728s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:35:05.992048021 (11h55m5.581393018s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:05.992051768 (11h55m5.581396755s) [upload=12393]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:35:35.339368736 (11h55m34.928713723s) [upload=12393]: Done uploading segments... Aug 7 21:35:35.339503534 (11h55m34.928848522s) [upload=12393]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:36:12.952326182 (11h56m12.541671159s) [upload=12411]: Starting upload Aug 7 21:36:12.952385075 (11h56m12.541730052s) [upload=12411]: Done starting upload Aug 7 21:36:12.952428238 (11h56m12.541773225s) [upload=12411]: Uploading segments... Aug 7 21:36:12.952455510 (11h56m12.541800497s) [upload=12411]: Got next segment. Inline: true Aug 7 21:36:12.952459417 (11h56m12.541804404s) [upload=12411]: Waiting for error group managing segments... Aug 7 21:36:12.952507650 (11h56m12.541852637s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:36:12.952514352 (11h56m12.541859349s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:12.952518550 (11h56m12.541863537s) [upload=12411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:36:42.299158316 (11h56m41.888503304s) [upload=12411]: Done uploading segments... Aug 7 21:36:42.299175039 (11h56m41.888520026s) [upload=12411]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded
Duration: 14.858404225s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [..................................X....X.X...X..............] duration:
773.271652ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:43:19 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 773.298884ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
36 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..] duration:
2.369703442s
FAIL (2025-08-07 21:43:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.369723551s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXXXXXXXXXXXX...] duration:
1m0.000700783s
FAIL (2025-08-07 21:43:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000713197s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
32 out of 60 is failed (threshold=3) [.......................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...] duration:
1.902620711s
FAIL (2025-08-07 21:43:13 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 1.902649596s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
26 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXXXXXXXXXX..] duration:
5.715155239s
FAIL (2025-08-07 21:43:10 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.715176368s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
25 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXXXXXXXXXXX....] duration:
17.328054267s
FAIL (2025-08-07 21:43:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
2975 (11h51m26.263997962s) [noise=true , address=104.36.82.45:4937 , piece_id=EBUAJKXBK6RZK7T46KXJT5P5LINBFVZCEPGVGG5OZ66SYNOA2RSQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: <nil>
Aug 7 21:31:26.680556480 (11h51m26.269901467s) [noise=true , address=45.91.116.10:4110 , piece_id=F4ZMNLCCFQCBFB6FOPOJFTGPIL5BFYT6TPGL5C7SVLIHMTY4EJJA, seg_pos={0 0}, upload=12321]: Done uploading piece. err: <nil>
Aug 7 21:31:26.680580406 (11h51m26.269925393s) [seg_pos={0 0}, upload=12321]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:31:26.680752626 (11h51m26.270097613s) [noise=true , address=45.91.116.6:4983 , piece_id=ZWH64KJWCVSTEGVN25WW35HRHTAO3BE3SPKKALNRT7NNLUKB4L4Q, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122trJbDHuqeBMQHZT97HP2PgkeuLyUiDiuRm4nayu6ZTH7tat6): protocol: expected piece hash; context canceled; EOF
Aug 7 21:31:26.680804515 (11h51m26.270149492s) [noise=true , address=45.91.116.7:4037 , piece_id=WY2F64PFXEGFVE4M6PUHHUYIMTI2KB4CB73DQADZK4QGIEZ2YCYQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK): protocol: expected piece hash; context canceled; EOF
Aug 7 21:31:26.680954802 (11h51m26.270299789s) [noise=true , address=104.36.82.61:4375 , piece_id=5LQOQ3MR3EA36DIYLOVO6YDKF6EP2HPHMPG4P6MN76A4IA5BTBEQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: write tcp 188.166.225.170:45604->104.36.82.61:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:31:26.680994198 (11h51m26.270339185s) [noise=true , address=104.36.82.8:4977 , piece_id=VY47HC4FF6JU4QQTDVZ62NOIEGWSROTWYMGCB7XXNHJ3GZNJATMQ, seg_pos={0 0}, upload=12321]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT): protocol: write tcp 188.166.225.170:36156->104.36.82.8:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 7 21:31:26.681016300 (11h51m26.270361287s) [seg_pos={0 0}, upload=12321]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:31:26.681067097 (11h51m26.270412074s) [upload=12321]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:31:26.681072137 (11h51m26.270417114s) [upload=12321]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:32:39.816785277 (11h52m39.406130264s) [upload=12321]: Done uploading segments...
Aug 7 21:32:39.816815415 (11h52m39.406160402s) [upload=12321]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:32:40.684664778 (11h52m40.274009755s) [upload=12355]: Starting upload
Aug 7 21:32:40.684721386 (11h52m40.274066373s) [upload=12355]: Done starting upload
Aug 7 21:32:40.684776000 (11h52m40.274121017s) [upload=12355]: Uploading segments...
Aug 7 21:32:40.684893314 (11h52m40.274238292s) [upload=12355]: Got next segment. Inline: false
Aug 7 21:32:40.684916699 (11h52m40.274261686s) [upload=12355]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:32:40.684920466 (11h52m40.274265443s) [upload=12355]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:33:40.036921304 (11h53m39.626266281s) [upload=12355]: Next segment err: context deadline exceeded
Aug 7 21:33:40.037128099 (11h53m39.626473086s) [upload=12355]: Done uploading segments...
Aug 7 21:33:40.037148919 (11h53m39.626493896s) [upload=12355]: Upload finished. err: context deadline exceeded
Aug 7 21:33:40.899438535 (11h53m40.488783522s) [upload=12372]: Starting upload
Aug 7 21:33:40.899490825 (11h53m40.488835832s) [upload=12372]: Done starting upload
Aug 7 21:33:40.899590144 (11h53m40.488935131s) [upload=12372]: Uploading segments...
Aug 7 21:33:40.899771832 (11h53m40.489116819s) [upload=12372]: Got next segment. Inline: false
Aug 7 21:33:40.899795227 (11h53m40.489140214s) [upload=12372]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:33:40.899799275 (11h53m40.489144252s) [upload=12372]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:34:40.257761531 (11h54m39.847106518s) [upload=12372]: Next segment err: context deadline exceeded
Aug 7 21:34:40.257885859 (11h54m39.847230856s) [upload=12372]: Done uploading segments...
Aug 7 21:34:40.257904695 (11h54m39.847249672s) [upload=12372]: Upload finished. err: context deadline exceeded
Aug 7 21:34:41.144124896 (11h54m40.733469873s) [upload=12386]: Starting upload
Aug 7 21:34:41.144233103 (11h54m40.73357813s) [upload=12386]: Done starting upload
Aug 7 21:34:41.144343113 (11h54m40.733688101s) [upload=12386]: Uploading segments...
Aug 7 21:34:41.144700337 (11h54m40.734045364s) [upload=12386]: Got next segment. Inline: false
Aug 7 21:34:41.144760612 (11h54m40.734105639s) [upload=12386]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:34:41.144768206 (11h54m40.734113223s) [upload=12386]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:35:40.491924814 (11h55m40.081269811s) [upload=12386]: Next segment err: context deadline exceeded
Aug 7 21:35:40.492048891 (11h55m40.081393878s) [upload=12386]: Done uploading segments...
Aug 7 21:35:40.492066364 (11h55m40.081411351s) [upload=12386]: Upload finished. err: context deadline exceeded
Aug 7 21:35:41.353263670 (11h55m40.942608657s) [upload=12405]: Starting upload
Aug 7 21:35:41.353369352 (11h55m40.942714339s) [upload=12405]: Done starting upload
Aug 7 21:35:41.353452601 (11h55m40.942797588s) [upload=12405]: Uploading segments...
Aug 7 21:35:41.353579473 (11h55m40.942924511s) [upload=12405]: Got next segment. Inline: false
Aug 7 21:35:41.353621805 (11h55m40.942966792s) [upload=12405]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:35:41.353626053 (11h55m40.94297104s) [upload=12405]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:36:40.713868241 (11h56m40.303213228s) [upload=12405]: Next segment err: context deadline exceeded
Aug 7 21:36:40.713958674 (11h56m40.303303651s) [upload=12405]: Done uploading segments...
Aug 7 21:36:40.713972460 (11h56m40.303317447s) [upload=12405]: Upload finished. err: context deadline exceeded
Aug 7 21:36:41.596698707 (11h56m41.186043744s) [upload=12418]: Starting upload
Aug 7 21:36:41.596808026 (11h56m41.186153063s) [upload=12418]: Done starting upload
Aug 7 21:36:41.596885475 (11h56m41.186230462s) [upload=12418]: Uploading segments...
Aug 7 21:36:41.597098722 (11h56m41.186443709s) [upload=12418]: Got next segment. Inline: false
Aug 7 21:36:41.597141925 (11h56m41.186486912s) [upload=12418]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:36:41.597146604 (11h56m41.186491591s) [upload=12418]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:37:40.947744552 (11h57m40.537089539s) [upload=12418]: Next segment err: context deadline exceeded
Aug 7 21:37:40.947825697 (11h57m40.537170684s) [upload=12418]: Done uploading segments...
Aug 7 21:37:40.947843592 (11h57m40.537188579s) [upload=12418]: Upload finished. err: context deadline exceeded
Aug 7 21:37:41.823629920 (11h57m41.412974907s) [upload=12437]: Starting upload
Aug 7 21:37:41.823680908 (11h57m41.413025895s) [upload=12437]: Done starting upload
Aug 7 21:37:41.823826305 (11h57m41.413171293s) [upload=12437]: Uploading segments...
Aug 7 21:37:41.823857055 (11h57m41.413202052s) [upload=12437]: Got next segment. Inline: false
Aug 7 21:37:41.823914255 (11h57m41.413259242s) [upload=12437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:37:41.823918503 (11h57m41.41326348s) [upload=12437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:38:41.179029912 (11h58m40.768374899s) [upload=12437]: Next segment err: context deadline exceeded
Aug 7 21:38:41.179128490 (11h58m40.768473477s) [upload=12437]: Done uploading segments...
Aug 7 21:38:41.179142979 (11h58m40.768487966s) [upload=12437]: Upload finished. err: context deadline exceeded
Aug 7 21:38:42.074438453 (11h58m41.66378344s) [upload=12451]: Starting upload
Aug 7 21:38:42.074516963 (11h58m41.66386195s) [upload=12451]: Done starting upload
Aug 7 21:38:42.074605202 (11h58m41.663950189s) [upload=12451]: Uploading segments...
Aug 7 21:38:42.074689222 (11h58m41.664034209s) [upload=12451]: Got next segment. Inline: false
Aug 7 21:38:42.074715102 (11h58m41.664060089s) [upload=12451]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:38:42.074718829 (11h58m41.664063816s) [upload=12451]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:39:41.428394360 (11h59m41.017739347s) [upload=12451]: Next segment err: context deadline exceeded
Aug 7 21:39:41.428496375 (11h59m41.017841372s) [upload=12451]: Done uploading segments...
Aug 7 21:39:41.428519450 (11h59m41.017864437s) [upload=12451]: Upload finished. err: context deadline exceeded
Aug 7 21:39:42.313134109 (11h59m41.902479096s) [upload=12468]: Starting upload
Aug 7 21:39:42.313191258 (11h59m41.902536245s) [upload=12468]: Done starting upload
Aug 7 21:39:42.313256903 (11h59m41.90260189s) [upload=12468]: Uploading segments...
Aug 7 21:39:42.313381803 (11h59m41.90272684s) [upload=12468]: Got next segment. Inline: false
Aug 7 21:39:42.313472736 (11h59m41.902817763s) [upload=12468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:39:42.313480300 (11h59m41.902825308s) [upload=12468]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:40:41.669512466 (12h0m41.258857454s) [upload=12468]: Next segment err: context deadline exceeded
Aug 7 21:40:41.669582440 (12h0m41.258927427s) [upload=12468]: Done uploading segments...
Aug 7 21:40:41.669600465 (12h0m41.258945452s) [upload=12468]: Upload finished. err: context deadline exceeded
Duration: 17.328085828s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
37 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.] duration:
28.571898567s
FAIL (2025-08-07 21:42:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 28.571913767s
edge
gateway
us-select-1
|