Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
720.58429ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:03:53 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 720.616822ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X...................X.....................................] duration:
7.518094048s
FAIL (2025-09-09 16:03:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.518119875s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.815554732s
FAIL (2025-09-09 16:03:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.815580541s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
53 out of 60 is failed (threshold=3) [.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.] duration:
24.425655933s
FAIL (2025-09-09 16:03:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 16:01:39.052720772 (74h33m15.092293902s) [upload=65596]: Starting upload
Sep 9 16:01:39.052821636 (74h33m15.092394756s) [upload=65596]: Done starting upload
Sep 9 16:01:39.052944691 (74h33m15.092517832s) [upload=65596]: Uploading segments...
Sep 9 16:01:39.053156787 (74h33m15.092729907s) [upload=65596]: Got next segment. Inline: false
Sep 9 16:01:39.053192466 (74h33m15.092765586s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 16:01:39.053198437 (74h33m15.092771557s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 16:01:39.326468308 (74h33m15.366041428s) [seg_pos={0 0}, upload=65596]: Begin upload segment...
Sep 9 16:01:39.326514125 (74h33m15.366087245s) [seg_pos={0 0}, upload=65596]: Done begin upload segment.
Sep 9 16:01:39.326537831 (74h33m15.366110961s) [noise=true , address=45.91.116.5:4297 , piece_id=7N6LCXMBD4CBYC3WAFD5MJYD4WOAYJMS534RIZR3AAHXGNULEKTQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.326576084 (74h33m15.366149214s) [noise=true , address=145.79.227.13:4801 , piece_id=LLD6SQRNDURT7GEI5PLP2XUHJLNSV53O54QRYX5F7E4VSCUIAJ7Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.326928388 (74h33m15.366501508s) [noise=true , address=45.91.116.10:4082 , piece_id=JSBYLAH3EZKIVJXAV32BDFS2I3H37MEGBAMBYYGRSK4LPWNJMIUA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327269992 (74h33m15.366843122s) [noise=true , address=77.90.31.14:4661 , piece_id=K77ZWOLKAJAX5HRA6IDEFCX6LYFSQ7ESCMSN4E2PRZ24NZMMQBTQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327296733 (74h33m15.366869934s) [noise=true , address=45.91.116.6:4973 , piece_id=OO7VFR3WRBT4F6YWPEBNEOHHAJ6YA2S45MPRKICITVTRZMEOGO3Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327917171 (74h33m15.367490301s) [noise=true , address=145.79.227.6:4258 , piece_id=GXHANM4U3F2TIOWPBQI2BCALOJHS5FZORQ76C6TF3SZBG4U3ULUQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327941227 (74h33m15.367514347s) [noise=true , address=45.91.116.12:4227 , piece_id=34Q2VRUXCAVPZFU7UDVPSEKQH65BM6HQJR6MFDJ5AFHGLJV3QJ7Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328288561 (74h33m15.367861681s) [noise=true , address=77.90.31.17:4772 , piece_id=KHSTHUBV6RLQMMW6GDIPZUYEWN2WGH6PH7TNM2DLXTZK7COEN62A, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328512871 (74h33m15.368085991s) [noise=true , address=77.90.31.10:4518 , piece_id=4V7VMPVIWGJJIHEN7XQC22L56OUTEFQ3KBILLU6WJLEVHZHFO6FQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328526456 (74h33m15.368099576s) [noise=true , address=45.91.116.8:4045 , piece_id=7HESKRLCSK3QY3IPZVTMNERCCVCNQPNTQ62OGWTWY24TOXDFH74Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328735156 (74h33m15.368308276s) [noise=true , address=77.90.31.12:4589 , piece_id=XWX7DLT7QTBHTA3SDKQI7OFTWCA2Z6UTPYPD5NBFXFCXMULY74IA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328929097 (74h33m15.368502217s) [noise=true , address=77.90.31.14:4663 , piece_id=Z7E7ON2LXLH3SCTUXTCQPJN4KEHAHMNF6OWFI6GLQR3B7KBK45JQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328959876 (74h33m15.368533246s) [noise=true , address=45.91.116.7:4009 , piece_id=MQAIISG33QI56MFLDJXZRT5PU5UF5WZNBASJMSRI3ASGUDBE7XTA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329239621 (74h33m15.368812751s) [noise=true , address=45.91.116.12:4225 , piece_id=R5UB6A66PHKNQIU4GK3JDU7OGOJHCUODRG7Q5PKETINHETOW7KMQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329355624 (74h33m15.368928754s) [noise=true , address=45.91.116.5:4298 , piece_id=62N7ZMCYHFVFMY4YYXCCTDIYB7YXSOCJ4ABBX6ZE5KVHB6HMNT3A, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329743616 (74h33m15.369316746s) [noise=true , address=145.79.227.7:4294 , piece_id=EUDT25VU4ZSIDVNDRT2HFDUFIYU3AQCNDRC2UMZ6EPAVTL36YWEA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:41.237643233 (74h33m17.277216354s) [upload=65596]: Next returned nil segment
Sep 9 16:01:41.237663331 (74h33m17.277236461s) [upload=65596]: Waiting for error group managing segments...
Sep 9 16:01:41.557699014 (74h33m17.597272134s) [noise=true , address=45.91.116.10:4082 , piece_id=JSBYLAH3EZKIVJXAV32BDFS2I3H37MEGBAMBYYGRSK4LPWNJMIUA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.616909177 (74h33m17.656482307s) [noise=true , address=145.79.227.6:4258 , piece_id=GXHANM4U3F2TIOWPBQI2BCALOJHS5FZORQ76C6TF3SZBG4U3ULUQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.650051418 (74h33m17.689624538s) [noise=true , address=45.91.116.5:4298 , piece_id=62N7ZMCYHFVFMY4YYXCCTDIYB7YXSOCJ4ABBX6ZE5KVHB6HMNT3A, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.653305625 (74h33m17.692878755s) [noise=true , address=77.90.31.14:4663 , piece_id=Z7E7ON2LXLH3SCTUXTCQPJN4KEHAHMNF6OWFI6GLQR3B7KBK45JQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.704706925 (74h33m17.744280045s) [noise=true , address=45.91.116.7:4009 , piece_id=MQAIISG33QI56MFLDJXZRT5PU5UF5WZNBASJMSRI3ASGUDBE7XTA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.753613140 (74h33m17.79318627s) [noise=true , address=45.91.116.5:4297 , piece_id=7N6LCXMBD4CBYC3WAFD5MJYD4WOAYJMS534RIZR3AAHXGNULEKTQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.765630406 (74h33m17.805203526s) [noise=true , address=77.90.31.14:4661 , piece_id=K77ZWOLKAJAX5HRA6IDEFCX6LYFSQ7ESCMSN4E2PRZ24NZMMQBTQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.781891374 (74h33m17.821464494s) [noise=true , address=145.79.227.13:4801 , piece_id=LLD6SQRNDURT7GEI5PLP2XUHJLNSV53O54QRYX5F7E4VSCUIAJ7Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.815242973 (74h33m17.854816093s) [noise=true , address=45.91.116.12:4227 , piece_id=34Q2VRUXCAVPZFU7UDVPSEKQH65BM6HQJR6MFDJ5AFHGLJV3QJ7Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.936354119 (74h33m17.97592724s) [noise=true , address=77.90.31.17:4772 , piece_id=KHSTHUBV6RLQMMW6GDIPZUYEWN2WGH6PH7TNM2DLXTZK7COEN62A, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.940907403 (74h33m17.980480523s) [noise=true , address=45.91.116.8:4045 , piece_id=7HESKRLCSK3QY3IPZVTMNERCCVCNQPNTQ62OGWTWY24TOXDFH74Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.055557234 (74h33m18.095130355s) [noise=true , address=45.91.116.6:4973 , piece_id=OO7VFR3WRBT4F6YWPEBNEOHHAJ6YA2S45MPRKICITVTRZMEOGO3Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.110682269 (74h33m18.150255399s) [noise=true , address=77.90.31.12:4589 , piece_id=XWX7DLT7QTBHTA3SDKQI7OFTWCA2Z6UTPYPD5NBFXFCXMULY74IA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.110715343 (74h33m18.150288473s) [seg_pos={0 0}, upload=65596]: Segment reached optimal threshold of 13 pieces.
Sep 9 16:01:42.110865560 (74h33m18.15043868s) [noise=true , address=145.79.227.7:4294 , piece_id=EUDT25VU4ZSIDVNDRT2HFDUFIYU3AQCNDRC2UMZ6EPAVTL36YWEA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF
Sep 9 16:01:42.110971203 (74h33m18.150544333s) [noise=true , address=77.90.31.10:4518 , piece_id=4V7VMPVIWGJJIHEN7XQC22L56OUTEFQ3KBILLU6WJLEVHZHFO6FQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Sep 9 16:01:42.111040386 (74h33m18.150613516s) [noise=true , address=45.91.116.12:4225 , piece_id=R5UB6A66PHKNQIU4GK3JDU7OGOJHCUODRG7Q5PKETINHETOW7KMQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection
Sep 9 16:01:42.111069661 (74h33m18.150642782s) [seg_pos={0 0}, upload=65596]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 16:01:42.111189711 (74h33m18.150762831s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 16:01:42.111195341 (74h33m18.150768461s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 16:01:42.593377332 (74h33m18.632950452s) [upload=65596]: Done uploading segments...
Sep 9 16:01:42.593396418 (74h33m18.632969538s) [upload=65596]: Upload finished. err: <nil>
Duration: 24.42567578s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
745.353579ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:02:57 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 745.377996ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X...................X....................................] duration:
7.39581601s
FAIL (2025-09-09 16:02:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.395841237s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.691209597s
FAIL (2025-09-09 16:02:32 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.69124776s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
54 out of 60 is failed (threshold=3) [X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X] duration:
30.002544848s
FAIL (2025-09-09 16:02:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 16:01:39.052720772 (74h33m15.092293902s) [upload=65596]: Starting upload
Sep 9 16:01:39.052821636 (74h33m15.092394756s) [upload=65596]: Done starting upload
Sep 9 16:01:39.052944691 (74h33m15.092517832s) [upload=65596]: Uploading segments...
Sep 9 16:01:39.053156787 (74h33m15.092729907s) [upload=65596]: Got next segment. Inline: false
Sep 9 16:01:39.053192466 (74h33m15.092765586s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 16:01:39.053198437 (74h33m15.092771557s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 16:01:39.326468308 (74h33m15.366041428s) [seg_pos={0 0}, upload=65596]: Begin upload segment...
Sep 9 16:01:39.326514125 (74h33m15.366087245s) [seg_pos={0 0}, upload=65596]: Done begin upload segment.
Sep 9 16:01:39.326537831 (74h33m15.366110961s) [noise=true , address=45.91.116.5:4297 , piece_id=7N6LCXMBD4CBYC3WAFD5MJYD4WOAYJMS534RIZR3AAHXGNULEKTQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.326576084 (74h33m15.366149214s) [noise=true , address=145.79.227.13:4801 , piece_id=LLD6SQRNDURT7GEI5PLP2XUHJLNSV53O54QRYX5F7E4VSCUIAJ7Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.326928388 (74h33m15.366501508s) [noise=true , address=45.91.116.10:4082 , piece_id=JSBYLAH3EZKIVJXAV32BDFS2I3H37MEGBAMBYYGRSK4LPWNJMIUA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327269992 (74h33m15.366843122s) [noise=true , address=77.90.31.14:4661 , piece_id=K77ZWOLKAJAX5HRA6IDEFCX6LYFSQ7ESCMSN4E2PRZ24NZMMQBTQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327296733 (74h33m15.366869934s) [noise=true , address=45.91.116.6:4973 , piece_id=OO7VFR3WRBT4F6YWPEBNEOHHAJ6YA2S45MPRKICITVTRZMEOGO3Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327917171 (74h33m15.367490301s) [noise=true , address=145.79.227.6:4258 , piece_id=GXHANM4U3F2TIOWPBQI2BCALOJHS5FZORQ76C6TF3SZBG4U3ULUQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.327941227 (74h33m15.367514347s) [noise=true , address=45.91.116.12:4227 , piece_id=34Q2VRUXCAVPZFU7UDVPSEKQH65BM6HQJR6MFDJ5AFHGLJV3QJ7Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328288561 (74h33m15.367861681s) [noise=true , address=77.90.31.17:4772 , piece_id=KHSTHUBV6RLQMMW6GDIPZUYEWN2WGH6PH7TNM2DLXTZK7COEN62A, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328512871 (74h33m15.368085991s) [noise=true , address=77.90.31.10:4518 , piece_id=4V7VMPVIWGJJIHEN7XQC22L56OUTEFQ3KBILLU6WJLEVHZHFO6FQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328526456 (74h33m15.368099576s) [noise=true , address=45.91.116.8:4045 , piece_id=7HESKRLCSK3QY3IPZVTMNERCCVCNQPNTQ62OGWTWY24TOXDFH74Q, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328735156 (74h33m15.368308276s) [noise=true , address=77.90.31.12:4589 , piece_id=XWX7DLT7QTBHTA3SDKQI7OFTWCA2Z6UTPYPD5NBFXFCXMULY74IA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328929097 (74h33m15.368502217s) [noise=true , address=77.90.31.14:4663 , piece_id=Z7E7ON2LXLH3SCTUXTCQPJN4KEHAHMNF6OWFI6GLQR3B7KBK45JQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.328959876 (74h33m15.368533246s) [noise=true , address=45.91.116.7:4009 , piece_id=MQAIISG33QI56MFLDJXZRT5PU5UF5WZNBASJMSRI3ASGUDBE7XTA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329239621 (74h33m15.368812751s) [noise=true , address=45.91.116.12:4225 , piece_id=R5UB6A66PHKNQIU4GK3JDU7OGOJHCUODRG7Q5PKETINHETOW7KMQ, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329355624 (74h33m15.368928754s) [noise=true , address=45.91.116.5:4298 , piece_id=62N7ZMCYHFVFMY4YYXCCTDIYB7YXSOCJ4ABBX6ZE5KVHB6HMNT3A, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:39.329743616 (74h33m15.369316746s) [noise=true , address=145.79.227.7:4294 , piece_id=EUDT25VU4ZSIDVNDRT2HFDUFIYU3AQCNDRC2UMZ6EPAVTL36YWEA, seg_pos={0 0}, upload=65596]: Uploading piece...
Sep 9 16:01:41.237643233 (74h33m17.277216354s) [upload=65596]: Next returned nil segment
Sep 9 16:01:41.237663331 (74h33m17.277236461s) [upload=65596]: Waiting for error group managing segments...
Sep 9 16:01:41.557699014 (74h33m17.597272134s) [noise=true , address=45.91.116.10:4082 , piece_id=JSBYLAH3EZKIVJXAV32BDFS2I3H37MEGBAMBYYGRSK4LPWNJMIUA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.616909177 (74h33m17.656482307s) [noise=true , address=145.79.227.6:4258 , piece_id=GXHANM4U3F2TIOWPBQI2BCALOJHS5FZORQ76C6TF3SZBG4U3ULUQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.650051418 (74h33m17.689624538s) [noise=true , address=45.91.116.5:4298 , piece_id=62N7ZMCYHFVFMY4YYXCCTDIYB7YXSOCJ4ABBX6ZE5KVHB6HMNT3A, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.653305625 (74h33m17.692878755s) [noise=true , address=77.90.31.14:4663 , piece_id=Z7E7ON2LXLH3SCTUXTCQPJN4KEHAHMNF6OWFI6GLQR3B7KBK45JQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.704706925 (74h33m17.744280045s) [noise=true , address=45.91.116.7:4009 , piece_id=MQAIISG33QI56MFLDJXZRT5PU5UF5WZNBASJMSRI3ASGUDBE7XTA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.753613140 (74h33m17.79318627s) [noise=true , address=45.91.116.5:4297 , piece_id=7N6LCXMBD4CBYC3WAFD5MJYD4WOAYJMS534RIZR3AAHXGNULEKTQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.765630406 (74h33m17.805203526s) [noise=true , address=77.90.31.14:4661 , piece_id=K77ZWOLKAJAX5HRA6IDEFCX6LYFSQ7ESCMSN4E2PRZ24NZMMQBTQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.781891374 (74h33m17.821464494s) [noise=true , address=145.79.227.13:4801 , piece_id=LLD6SQRNDURT7GEI5PLP2XUHJLNSV53O54QRYX5F7E4VSCUIAJ7Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.815242973 (74h33m17.854816093s) [noise=true , address=45.91.116.12:4227 , piece_id=34Q2VRUXCAVPZFU7UDVPSEKQH65BM6HQJR6MFDJ5AFHGLJV3QJ7Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.936354119 (74h33m17.97592724s) [noise=true , address=77.90.31.17:4772 , piece_id=KHSTHUBV6RLQMMW6GDIPZUYEWN2WGH6PH7TNM2DLXTZK7COEN62A, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:41.940907403 (74h33m17.980480523s) [noise=true , address=45.91.116.8:4045 , piece_id=7HESKRLCSK3QY3IPZVTMNERCCVCNQPNTQ62OGWTWY24TOXDFH74Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.055557234 (74h33m18.095130355s) [noise=true , address=45.91.116.6:4973 , piece_id=OO7VFR3WRBT4F6YWPEBNEOHHAJ6YA2S45MPRKICITVTRZMEOGO3Q, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.110682269 (74h33m18.150255399s) [noise=true , address=77.90.31.12:4589 , piece_id=XWX7DLT7QTBHTA3SDKQI7OFTWCA2Z6UTPYPD5NBFXFCXMULY74IA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: <nil>
Sep 9 16:01:42.110715343 (74h33m18.150288473s) [seg_pos={0 0}, upload=65596]: Segment reached optimal threshold of 13 pieces.
Sep 9 16:01:42.110865560 (74h33m18.15043868s) [noise=true , address=145.79.227.7:4294 , piece_id=EUDT25VU4ZSIDVNDRT2HFDUFIYU3AQCNDRC2UMZ6EPAVTL36YWEA, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF
Sep 9 16:01:42.110971203 (74h33m18.150544333s) [noise=true , address=77.90.31.10:4518 , piece_id=4V7VMPVIWGJJIHEN7XQC22L56OUTEFQ3KBILLU6WJLEVHZHFO6FQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Sep 9 16:01:42.111040386 (74h33m18.150613516s) [noise=true , address=45.91.116.12:4225 , piece_id=R5UB6A66PHKNQIU4GK3JDU7OGOJHCUODRG7Q5PKETINHETOW7KMQ, seg_pos={0 0}, upload=65596]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36650->45.91.116.12:4225: use of closed network connection
Sep 9 16:01:42.111069661 (74h33m18.150642782s) [seg_pos={0 0}, upload=65596]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 16:01:42.111189711 (74h33m18.150762831s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 16:01:42.111195341 (74h33m18.150768461s) [upload=65596]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 16:01:42.593377332 (74h33m18.632950452s) [upload=65596]: Done uploading segments...
Sep 9 16:01:42.593396418 (74h33m18.632969538s) [upload=65596]: Upload finished. err: <nil>
Duration: 30.002569195s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
762.211457ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:02:04 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 762.239511ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X...................X...................................] duration:
7.866243459s
FAIL (2025-09-09 16:01:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.866297853s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.546026107s
FAIL (2025-09-09 16:01:36 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.546046114s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
53 out of 60 is failed (threshold=3) [.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.] duration:
26.583554721s
FAIL (2025-09-09 16:01:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 16:00:02.802052854 (74h31m38.841625994s) [upload=65575]: Starting upload
Sep 9 16:00:02.802138909 (74h31m38.841712049s) [upload=65575]: Done starting upload
Sep 9 16:00:02.802375170 (74h31m38.841948311s) [upload=65575]: Uploading segments...
Sep 9 16:00:02.802425777 (74h31m38.841998908s) [upload=65575]: Got next segment. Inline: false
Sep 9 16:00:02.802472136 (74h31m38.842045267s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 16:00:02.802478859 (74h31m38.842051989s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 16:00:03.056588658 (74h31m39.096161788s) [seg_pos={0 0}, upload=65575]: Begin upload segment...
Sep 9 16:00:03.056654554 (74h31m39.096227684s) [seg_pos={0 0}, upload=65575]: Done begin upload segment.
Sep 9 16:00:03.056687507 (74h31m39.096260637s) [noise=true , address=145.79.227.15:4875 , piece_id=O4Q2VJ7X6KT7O7KQ66I6I5RW3NSLEPJRQE3CMWMSRPGGNBJ7UZXA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.056699129 (74h31m39.096272269s) [noise=true , address=45.91.116.8:4047 , piece_id=5THB7WMFCMYQLNTMKBQF2ZBT6UWZK4F3HQI26D4FQLRR55PAHEOA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057176683 (74h31m39.096749813s) [noise=true , address=77.90.31.17:4772 , piece_id=34EKYUIFZP4VQ23NC7IJ2BJV2ZMYVIUXWHOEWJFSROFRSNERAVTA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057193004 (74h31m39.096766144s) [noise=true , address=45.91.116.11:4119 , piece_id=F74UAIXFW2NSAJF4AZ6PDCDLM3QKSN7F6TVI2QLFHOQHRAG6GZCA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057518477 (74h31m39.097091607s) [noise=true , address=77.90.31.14:4662 , piece_id=442GYOFXJO7OZU6DHDNUIQHRWCXKM3LYMLIEVXDD4JCSPLSRGLLQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057869459 (74h31m39.097442589s) [noise=true , address=45.91.116.8:4045 , piece_id=LRXMJPCROAKBKIN5JHKFTO2QLQ6QFH4UWIEYDQFM2HFLZEMOJUQQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058192577 (74h31m39.097765707s) [noise=true , address=145.79.227.5:4223 , piece_id=3IOBIVKGYXREUHYMKWM3BIWGP5VJ5F5CIYD32JX6EIFCXHQ6KIJA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058511798 (74h31m39.098084928s) [noise=true , address=45.91.116.9:4262 , piece_id=L4E63NYGUPXCSFF7V3Z25IMZJOVLXKUAEQBFHGHKW3XHUFSAT5WA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058802324 (74h31m39.098375444s) [noise=true , address=77.90.31.21:4878 , piece_id=DKH42LZV3YDDYWLATD4RW4TIFMYLPJGLEDUGEYTZAQPE3JCAY24Q, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059069575 (74h31m39.098642705s) [noise=true , address=77.90.31.21:4877 , piece_id=RGIFY3PQKZPID2HZC2KDRW5BR4HNH3PPFHEYV2TL72JZD5AHUCJQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059390570 (74h31m39.0989637s) [noise=true , address=45.91.116.14:4190 , piece_id=SI4V24UX4VET3F7IRXCOSWEJMVHPP6RY3K6SYAJKC4EUJG3XGQAQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059817436 (74h31m39.099390566s) [noise=true , address=77.90.31.10:4518 , piece_id=G6DSM6YFQXLXLAPMIBFXDLH7N2KQYTFCXVSXYLOCZUQ6JBELANJQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060173397 (74h31m39.099746527s) [noise=true , address=77.90.31.13:4628 , piece_id=CHMDYY2OMWOQY5BBIPV33HJLZ4MIJCPRXX3ZX2J7RPWYFRXGU3GQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060487128 (74h31m39.100060258s) [noise=true , address=45.91.116.7:4011 , piece_id=6XQ2ADB33X72ULONXUU3JGM4UIVUWUIJ25TQCAEF3HKFU5NALGOA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060770029 (74h31m39.100343159s) [noise=true , address=145.79.227.17:4970 , piece_id=2VK4ZAYU6DHX5B5VV6U3UQPTJHD672F7ANCDHFFKYCSF6R4FHJIA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.061055235 (74h31m39.100628365s) [noise=true , address=45.91.116.6:4974 , piece_id=3WXDK7DA5DUTKCU447C44DX6WEZPEAJ5EUA6XQAU4QZ65TGDXVRA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:04.932763267 (74h31m40.972336397s) [upload=65575]: Next returned nil segment
Sep 9 16:00:04.932781872 (74h31m40.972354993s) [upload=65575]: Waiting for error group managing segments...
Sep 9 16:00:05.286117105 (74h31m41.325690235s) [noise=true , address=45.91.116.11:4119 , piece_id=F74UAIXFW2NSAJF4AZ6PDCDLM3QKSN7F6TVI2QLFHOQHRAG6GZCA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.298542964 (74h31m41.338116094s) [noise=true , address=45.91.116.6:4974 , piece_id=3WXDK7DA5DUTKCU447C44DX6WEZPEAJ5EUA6XQAU4QZ65TGDXVRA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.304947789 (74h31m41.344520919s) [noise=true , address=45.91.116.14:4190 , piece_id=SI4V24UX4VET3F7IRXCOSWEJMVHPP6RY3K6SYAJKC4EUJG3XGQAQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.326659196 (74h31m41.366232316s) [noise=true , address=45.91.116.8:4047 , piece_id=5THB7WMFCMYQLNTMKBQF2ZBT6UWZK4F3HQI26D4FQLRR55PAHEOA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.337299447 (74h31m41.376872597s) [noise=true , address=45.91.116.7:4011 , piece_id=6XQ2ADB33X72ULONXUU3JGM4UIVUWUIJ25TQCAEF3HKFU5NALGOA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.338160775 (74h31m41.377733915s) [noise=true , address=77.90.31.21:4877 , piece_id=RGIFY3PQKZPID2HZC2KDRW5BR4HNH3PPFHEYV2TL72JZD5AHUCJQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.367245591 (74h31m41.406818721s) [noise=true , address=45.91.116.9:4262 , piece_id=L4E63NYGUPXCSFF7V3Z25IMZJOVLXKUAEQBFHGHKW3XHUFSAT5WA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.372620124 (74h31m41.412193255s) [noise=true , address=45.91.116.8:4045 , piece_id=LRXMJPCROAKBKIN5JHKFTO2QLQ6QFH4UWIEYDQFM2HFLZEMOJUQQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.423319041 (74h31m41.462892171s) [noise=true , address=77.90.31.14:4662 , piece_id=442GYOFXJO7OZU6DHDNUIQHRWCXKM3LYMLIEVXDD4JCSPLSRGLLQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.423358707 (74h31m41.462931837s) [noise=true , address=77.90.31.21:4878 , piece_id=DKH42LZV3YDDYWLATD4RW4TIFMYLPJGLEDUGEYTZAQPE3JCAY24Q, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.459262241 (74h31m41.498835371s) [noise=true , address=145.79.227.17:4970 , piece_id=2VK4ZAYU6DHX5B5VV6U3UQPTJHD672F7ANCDHFFKYCSF6R4FHJIA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.474309455 (74h31m41.513882575s) [noise=true , address=77.90.31.10:4518 , piece_id=G6DSM6YFQXLXLAPMIBFXDLH7N2KQYTFCXVSXYLOCZUQ6JBELANJQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.842704977 (74h31m41.882278107s) [noise=true , address=77.90.31.17:4772 , piece_id=34EKYUIFZP4VQ23NC7IJ2BJV2ZMYVIUXWHOEWJFSROFRSNERAVTA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.842754461 (74h31m41.882327581s) [seg_pos={0 0}, upload=65575]: Segment reached optimal threshold of 13 pieces.
Sep 9 16:00:05.842986285 (74h31m41.882559415s) [noise=true , address=145.79.227.15:4875 , piece_id=O4Q2VJ7X6KT7O7KQ66I6I5RW3NSLEPJRQE3CMWMSRPGGNBJ7UZXA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843087539 (74h31m41.882660669s) [noise=true , address=145.79.227.5:4223 , piece_id=3IOBIVKGYXREUHYMKWM3BIWGP5VJ5F5CIYD32JX6EIFCXHQ6KIJA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843174385 (74h31m41.882747515s) [noise=true , address=77.90.31.13:4628 , piece_id=CHMDYY2OMWOQY5BBIPV33HJLZ4MIJCPRXX3ZX2J7RPWYFRXGU3GQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843239129 (74h31m41.882812259s) [seg_pos={0 0}, upload=65575]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 16:00:05.843336205 (74h31m41.882909335s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 16:00:05.843342567 (74h31m41.882915697s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 16:00:06.163423946 (74h31m42.202997087s) [upload=65575]: Done uploading segments...
Sep 9 16:00:06.163449405 (74h31m42.203022525s) [upload=65575]: Upload finished. err: <nil>
Duration: 26.583582154s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
742.019288ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:01:14 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 742.057922ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [X....X...................X..................................] duration:
7.133146219s
FAIL (2025-09-09 16:00:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.133174043s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.84771441s
FAIL (2025-09-09 16:00:43 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.847834991s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X....X...................X.................................] duration:
6.547532345s
FAIL (2025-09-09 16:00:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.547560418s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
54 out of 60 is failed (threshold=3) [X.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X] duration:
30.002509364s
FAIL (2025-09-09 16:00:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 16:00:02.802052854 (74h31m38.841625994s) [upload=65575]: Starting upload
Sep 9 16:00:02.802138909 (74h31m38.841712049s) [upload=65575]: Done starting upload
Sep 9 16:00:02.802375170 (74h31m38.841948311s) [upload=65575]: Uploading segments...
Sep 9 16:00:02.802425777 (74h31m38.841998908s) [upload=65575]: Got next segment. Inline: false
Sep 9 16:00:02.802472136 (74h31m38.842045267s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 16:00:02.802478859 (74h31m38.842051989s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 16:00:03.056588658 (74h31m39.096161788s) [seg_pos={0 0}, upload=65575]: Begin upload segment...
Sep 9 16:00:03.056654554 (74h31m39.096227684s) [seg_pos={0 0}, upload=65575]: Done begin upload segment.
Sep 9 16:00:03.056687507 (74h31m39.096260637s) [noise=true , address=145.79.227.15:4875 , piece_id=O4Q2VJ7X6KT7O7KQ66I6I5RW3NSLEPJRQE3CMWMSRPGGNBJ7UZXA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.056699129 (74h31m39.096272269s) [noise=true , address=45.91.116.8:4047 , piece_id=5THB7WMFCMYQLNTMKBQF2ZBT6UWZK4F3HQI26D4FQLRR55PAHEOA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057176683 (74h31m39.096749813s) [noise=true , address=77.90.31.17:4772 , piece_id=34EKYUIFZP4VQ23NC7IJ2BJV2ZMYVIUXWHOEWJFSROFRSNERAVTA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057193004 (74h31m39.096766144s) [noise=true , address=45.91.116.11:4119 , piece_id=F74UAIXFW2NSAJF4AZ6PDCDLM3QKSN7F6TVI2QLFHOQHRAG6GZCA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057518477 (74h31m39.097091607s) [noise=true , address=77.90.31.14:4662 , piece_id=442GYOFXJO7OZU6DHDNUIQHRWCXKM3LYMLIEVXDD4JCSPLSRGLLQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.057869459 (74h31m39.097442589s) [noise=true , address=45.91.116.8:4045 , piece_id=LRXMJPCROAKBKIN5JHKFTO2QLQ6QFH4UWIEYDQFM2HFLZEMOJUQQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058192577 (74h31m39.097765707s) [noise=true , address=145.79.227.5:4223 , piece_id=3IOBIVKGYXREUHYMKWM3BIWGP5VJ5F5CIYD32JX6EIFCXHQ6KIJA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058511798 (74h31m39.098084928s) [noise=true , address=45.91.116.9:4262 , piece_id=L4E63NYGUPXCSFF7V3Z25IMZJOVLXKUAEQBFHGHKW3XHUFSAT5WA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.058802324 (74h31m39.098375444s) [noise=true , address=77.90.31.21:4878 , piece_id=DKH42LZV3YDDYWLATD4RW4TIFMYLPJGLEDUGEYTZAQPE3JCAY24Q, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059069575 (74h31m39.098642705s) [noise=true , address=77.90.31.21:4877 , piece_id=RGIFY3PQKZPID2HZC2KDRW5BR4HNH3PPFHEYV2TL72JZD5AHUCJQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059390570 (74h31m39.0989637s) [noise=true , address=45.91.116.14:4190 , piece_id=SI4V24UX4VET3F7IRXCOSWEJMVHPP6RY3K6SYAJKC4EUJG3XGQAQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.059817436 (74h31m39.099390566s) [noise=true , address=77.90.31.10:4518 , piece_id=G6DSM6YFQXLXLAPMIBFXDLH7N2KQYTFCXVSXYLOCZUQ6JBELANJQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060173397 (74h31m39.099746527s) [noise=true , address=77.90.31.13:4628 , piece_id=CHMDYY2OMWOQY5BBIPV33HJLZ4MIJCPRXX3ZX2J7RPWYFRXGU3GQ, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060487128 (74h31m39.100060258s) [noise=true , address=45.91.116.7:4011 , piece_id=6XQ2ADB33X72ULONXUU3JGM4UIVUWUIJ25TQCAEF3HKFU5NALGOA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.060770029 (74h31m39.100343159s) [noise=true , address=145.79.227.17:4970 , piece_id=2VK4ZAYU6DHX5B5VV6U3UQPTJHD672F7ANCDHFFKYCSF6R4FHJIA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:03.061055235 (74h31m39.100628365s) [noise=true , address=45.91.116.6:4974 , piece_id=3WXDK7DA5DUTKCU447C44DX6WEZPEAJ5EUA6XQAU4QZ65TGDXVRA, seg_pos={0 0}, upload=65575]: Uploading piece...
Sep 9 16:00:04.932763267 (74h31m40.972336397s) [upload=65575]: Next returned nil segment
Sep 9 16:00:04.932781872 (74h31m40.972354993s) [upload=65575]: Waiting for error group managing segments...
Sep 9 16:00:05.286117105 (74h31m41.325690235s) [noise=true , address=45.91.116.11:4119 , piece_id=F74UAIXFW2NSAJF4AZ6PDCDLM3QKSN7F6TVI2QLFHOQHRAG6GZCA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.298542964 (74h31m41.338116094s) [noise=true , address=45.91.116.6:4974 , piece_id=3WXDK7DA5DUTKCU447C44DX6WEZPEAJ5EUA6XQAU4QZ65TGDXVRA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.304947789 (74h31m41.344520919s) [noise=true , address=45.91.116.14:4190 , piece_id=SI4V24UX4VET3F7IRXCOSWEJMVHPP6RY3K6SYAJKC4EUJG3XGQAQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.326659196 (74h31m41.366232316s) [noise=true , address=45.91.116.8:4047 , piece_id=5THB7WMFCMYQLNTMKBQF2ZBT6UWZK4F3HQI26D4FQLRR55PAHEOA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.337299447 (74h31m41.376872597s) [noise=true , address=45.91.116.7:4011 , piece_id=6XQ2ADB33X72ULONXUU3JGM4UIVUWUIJ25TQCAEF3HKFU5NALGOA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.338160775 (74h31m41.377733915s) [noise=true , address=77.90.31.21:4877 , piece_id=RGIFY3PQKZPID2HZC2KDRW5BR4HNH3PPFHEYV2TL72JZD5AHUCJQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.367245591 (74h31m41.406818721s) [noise=true , address=45.91.116.9:4262 , piece_id=L4E63NYGUPXCSFF7V3Z25IMZJOVLXKUAEQBFHGHKW3XHUFSAT5WA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.372620124 (74h31m41.412193255s) [noise=true , address=45.91.116.8:4045 , piece_id=LRXMJPCROAKBKIN5JHKFTO2QLQ6QFH4UWIEYDQFM2HFLZEMOJUQQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.423319041 (74h31m41.462892171s) [noise=true , address=77.90.31.14:4662 , piece_id=442GYOFXJO7OZU6DHDNUIQHRWCXKM3LYMLIEVXDD4JCSPLSRGLLQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.423358707 (74h31m41.462931837s) [noise=true , address=77.90.31.21:4878 , piece_id=DKH42LZV3YDDYWLATD4RW4TIFMYLPJGLEDUGEYTZAQPE3JCAY24Q, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.459262241 (74h31m41.498835371s) [noise=true , address=145.79.227.17:4970 , piece_id=2VK4ZAYU6DHX5B5VV6U3UQPTJHD672F7ANCDHFFKYCSF6R4FHJIA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.474309455 (74h31m41.513882575s) [noise=true , address=77.90.31.10:4518 , piece_id=G6DSM6YFQXLXLAPMIBFXDLH7N2KQYTFCXVSXYLOCZUQ6JBELANJQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.842704977 (74h31m41.882278107s) [noise=true , address=77.90.31.17:4772 , piece_id=34EKYUIFZP4VQ23NC7IJ2BJV2ZMYVIUXWHOEWJFSROFRSNERAVTA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: <nil>
Sep 9 16:00:05.842754461 (74h31m41.882327581s) [seg_pos={0 0}, upload=65575]: Segment reached optimal threshold of 13 pieces.
Sep 9 16:00:05.842986285 (74h31m41.882559415s) [noise=true , address=145.79.227.15:4875 , piece_id=O4Q2VJ7X6KT7O7KQ66I6I5RW3NSLEPJRQE3CMWMSRPGGNBJ7UZXA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843087539 (74h31m41.882660669s) [noise=true , address=145.79.227.5:4223 , piece_id=3IOBIVKGYXREUHYMKWM3BIWGP5VJ5F5CIYD32JX6EIFCXHQ6KIJA, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843174385 (74h31m41.882747515s) [noise=true , address=77.90.31.13:4628 , piece_id=CHMDYY2OMWOQY5BBIPV33HJLZ4MIJCPRXX3ZX2J7RPWYFRXGU3GQ, seg_pos={0 0}, upload=65575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF
Sep 9 16:00:05.843239129 (74h31m41.882812259s) [seg_pos={0 0}, upload=65575]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 16:00:05.843336205 (74h31m41.882909335s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 16:00:05.843342567 (74h31m41.882915697s) [upload=65575]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 16:00:06.163423946 (74h31m42.202997087s) [upload=65575]: Done uploading segments...
Sep 9 16:00:06.163449405 (74h31m42.203022525s) [upload=65575]: Upload finished. err: <nil>
Duration: 30.002523212s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
701.869359ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:00:17 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 701.898846ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
54 out of 60 is failed (threshold=3) [XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.] duration:
29.424186869s
FAIL (2025-09-09 16:00:01 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Sep 9 15:58:40.741129715 (74h30m16.780702835s) [upload=65553]: Starting upload
Sep 9 15:58:40.741189320 (74h30m16.78076244s) [upload=65553]: Done starting upload
Sep 9 15:58:40.741344266 (74h30m16.780917396s) [upload=65553]: Uploading segments...
Sep 9 15:58:40.741464276 (74h30m16.781037396s) [upload=65553]: Got next segment. Inline: false
Sep 9 15:58:40.741543558 (74h30m16.781116718s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:58:40.741555260 (74h30m16.78112842s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:58:41.024235510 (74h30m17.06380863s) [seg_pos={0 0}, upload=65553]: Begin upload segment...
Sep 9 15:58:41.024297649 (74h30m17.063870769s) [seg_pos={0 0}, upload=65553]: Done begin upload segment.
Sep 9 15:58:41.024339950 (74h30m17.06391307s) [noise=true , address=77.90.31.14:4661 , piece_id=LEX3JXQO6JIFJSE3SKQ2XGKQOSWXZSKUPQHQWH5L7U5BKFI6JQ5Q, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024361632 (74h30m17.063934752s) [noise=true , address=45.91.116.11:4117 , piece_id=SF6577AWKAHTDELU4P7DNNUG4FJR76GYQXQ4NBB2PMDISNUE7NWQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024804689 (74h30m17.064377819s) [noise=true , address=77.90.31.14:4662 , piece_id=WTAAIECILA5LSJWCB7Y7PORZZHQQ2OR5W2WWTKUTEX2J42NLDVLA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024842732 (74h30m17.064415852s) [noise=true , address=77.90.31.12:4590 , piece_id=AZGNWLAKPTZK4OQJWFJK452VKSYFWHRXHKDTWMYH37G5S3MJVWZA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025156733 (74h30m17.064729863s) [noise=true , address=77.90.31.21:4878 , piece_id=A7XD5IZ47PHHUML6IP7UC7J7ZBSV55223LLWQHHTIWTGYM2ERXSA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025501453 (74h30m17.065074573s) [noise=true , address=77.90.31.10:4517 , piece_id=SV3AGAFN6VD5PX6UAJDSMFGMPAL6JL4KQY3MLMAVJRLJ3VKAAQQQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025837175 (74h30m17.065410295s) [noise=true , address=45.91.116.11:4118 , piece_id=55ZVFGBOPFGMEFXCJI2ZLO6JVXT5QIP4MDOUBR3GHGQN2Q5Z2P3A, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026152899 (74h30m17.065726029s) [noise=true , address=45.91.116.7:4009 , piece_id=3UA5Q4YZSM5KEYLBFBVI2VRDESGB7CEGLAVDVMA5NYOUEZACI6XA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026467651 (74h30m17.066040772s) [noise=true , address=45.91.116.6:4975 , piece_id=7PISBWRCIAYWKGKLECRHDDR6R7SJ4RNQCZ4QQMXSYXU4V6E2WU2Q, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026815467 (74h30m17.066388587s) [noise=true , address=77.90.31.17:4771 , piece_id=SDG7FI2WX2YW6NK6AOI3M3ZF3GMZXOFLRSHQJA4ARAQQHTWSVDRQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027174484 (74h30m17.066747604s) [noise=true , address=77.90.31.21:4879 , piece_id=YTTOURVEYHWVW3KJ3X3S7C5RDMXS7SEQWRHFI7W77672RNELIPEQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027423050 (74h30m17.06699617s) [noise=true , address=45.91.116.14:4189 , piece_id=MEXH26MGR623N44EOKXZAKMR6BPBAXWKWDLNRP5XEDO5QLXHX3SA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027788729 (74h30m17.067361859s) [noise=true , address=77.90.31.15:4697 , piece_id=OUBA3X3KXDVMTZCTFJUUFUTGE7S5TXD7DUBCFI2KJLFDQCMSQ3QQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028096337 (74h30m17.067669457s) [noise=true , address=45.91.116.8:4048 , piece_id=7S5POEBLCZF6JE53OSANSHNFXVKYMCJR6HX2UIVFM26ZSKU7SFYQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028407233 (74h30m17.067980353s) [noise=true , address=77.90.31.18:4805 , piece_id=2CQVXRFXHMY35FXVZEDTF5E23O4MLQZTS4VVYJ2FWSAFTTJNNOYQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028571768 (74h30m17.068144888s) [noise=true , address=45.91.116.5:4297 , piece_id=HAP64MIMSQWN5ROH6HIMCC6B5DTCHG6XKXUBZXAOAFSXCGHFQ2HA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:42.880051068 (74h30m18.919624188s) [upload=65553]: Next returned nil segment
Sep 9 15:58:42.880074704 (74h30m18.919647904s) [upload=65553]: Waiting for error group managing segments...
Sep 9 15:58:43.169368198 (74h30m19.208941318s) [noise=true , address=77.90.31.21:4879 , piece_id=YTTOURVEYHWVW3KJ3X3S7C5RDMXS7SEQWRHFI7W77672RNELIPEQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.221767237 (74h30m19.261340357s) [noise=true , address=77.90.31.17:4771 , piece_id=SDG7FI2WX2YW6NK6AOI3M3ZF3GMZXOFLRSHQJA4ARAQQHTWSVDRQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.245871151 (74h30m19.285444261s) [noise=true , address=77.90.31.10:4517 , piece_id=SV3AGAFN6VD5PX6UAJDSMFGMPAL6JL4KQY3MLMAVJRLJ3VKAAQQQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.253603979 (74h30m19.293177099s) [noise=true , address=45.91.116.6:4975 , piece_id=7PISBWRCIAYWKGKLECRHDDR6R7SJ4RNQCZ4QQMXSYXU4V6E2WU2Q, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.267280440 (74h30m19.30685358s) [noise=true , address=45.91.116.7:4009 , piece_id=3UA5Q4YZSM5KEYLBFBVI2VRDESGB7CEGLAVDVMA5NYOUEZACI6XA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.268819054 (74h30m19.308392184s) [noise=true , address=77.90.31.14:4661 , piece_id=LEX3JXQO6JIFJSE3SKQ2XGKQOSWXZSKUPQHQWH5L7U5BKFI6JQ5Q, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.279705877 (74h30m19.319278997s) [noise=true , address=77.90.31.14:4662 , piece_id=WTAAIECILA5LSJWCB7Y7PORZZHQQ2OR5W2WWTKUTEX2J42NLDVLA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.318054870 (74h30m19.35762799s) [noise=true , address=77.90.31.12:4590 , piece_id=AZGNWLAKPTZK4OQJWFJK452VKSYFWHRXHKDTWMYH37G5S3MJVWZA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.355173029 (74h30m19.394746149s) [noise=true , address=77.90.31.21:4878 , piece_id=A7XD5IZ47PHHUML6IP7UC7J7ZBSV55223LLWQHHTIWTGYM2ERXSA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.521692446 (74h30m19.561265576s) [noise=true , address=45.91.116.5:4297 , piece_id=HAP64MIMSQWN5ROH6HIMCC6B5DTCHG6XKXUBZXAOAFSXCGHFQ2HA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.550762924 (74h30m19.590336044s) [noise=true , address=77.90.31.18:4805 , piece_id=2CQVXRFXHMY35FXVZEDTF5E23O4MLQZTS4VVYJ2FWSAFTTJNNOYQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.617186440 (74h30m19.65675957s) [noise=true , address=45.91.116.14:4189 , piece_id=MEXH26MGR623N44EOKXZAKMR6BPBAXWKWDLNRP5XEDO5QLXHX3SA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.628619508 (74h30m19.668192629s) [noise=true , address=77.90.31.15:4697 , piece_id=OUBA3X3KXDVMTZCTFJUUFUTGE7S5TXD7DUBCFI2KJLFDQCMSQ3QQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.628645568 (74h30m19.668218688s) [seg_pos={0 0}, upload=65553]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:58:43.628879677 (74h30m19.668452797s) [noise=true , address=45.91.116.11:4118 , piece_id=55ZVFGBOPFGMEFXCJI2ZLO6JVXT5QIP4MDOUBR3GHGQN2Q5Z2P3A, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection
Sep 9 15:58:43.628921888 (74h30m19.668495008s) [noise=true , address=45.91.116.11:4117 , piece_id=SF6577AWKAHTDELU4P7DNNUG4FJR76GYQXQ4NBB2PMDISNUE7NWQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection; write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection
Sep 9 15:58:43.628979578 (74h30m19.668552698s) [noise=true , address=45.91.116.8:4048 , piece_id=7S5POEBLCZF6JE53OSANSHNFXVKYMCJR6HX2UIVFM26ZSKU7SFYQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF
Sep 9 15:58:43.629039723 (74h30m19.668612843s) [seg_pos={0 0}, upload=65553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:58:43.629138882 (74h30m19.668712012s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:58:43.629144594 (74h30m19.668717714s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:58:43.968648710 (74h30m20.00822184s) [upload=65553]: Done uploading segments...
Sep 9 15:58:43.968671363 (74h30m20.008244533s) [upload=65553]: Upload finished. err: <nil>
Duration: 29.424216495s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
680.73168ms target:
https://us1.storj.io/login
FAIL (2025-09-09 16:00:00 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 680.746778ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
3.121014211s
FAIL (2025-09-09 15:59:39 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.121036534s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [X.X....X...................X................................] duration:
7.209348424s
FAIL (2025-09-09 15:59:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.209378673s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
54 out of 60 is failed (threshold=3) [.XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.002665429s
FAIL (2025-09-09 15:59:09 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Sep 9 15:58:40.741129715 (74h30m16.780702835s) [upload=65553]: Starting upload
Sep 9 15:58:40.741189320 (74h30m16.78076244s) [upload=65553]: Done starting upload
Sep 9 15:58:40.741344266 (74h30m16.780917396s) [upload=65553]: Uploading segments...
Sep 9 15:58:40.741464276 (74h30m16.781037396s) [upload=65553]: Got next segment. Inline: false
Sep 9 15:58:40.741543558 (74h30m16.781116718s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:58:40.741555260 (74h30m16.78112842s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:58:41.024235510 (74h30m17.06380863s) [seg_pos={0 0}, upload=65553]: Begin upload segment...
Sep 9 15:58:41.024297649 (74h30m17.063870769s) [seg_pos={0 0}, upload=65553]: Done begin upload segment.
Sep 9 15:58:41.024339950 (74h30m17.06391307s) [noise=true , address=77.90.31.14:4661 , piece_id=LEX3JXQO6JIFJSE3SKQ2XGKQOSWXZSKUPQHQWH5L7U5BKFI6JQ5Q, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024361632 (74h30m17.063934752s) [noise=true , address=45.91.116.11:4117 , piece_id=SF6577AWKAHTDELU4P7DNNUG4FJR76GYQXQ4NBB2PMDISNUE7NWQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024804689 (74h30m17.064377819s) [noise=true , address=77.90.31.14:4662 , piece_id=WTAAIECILA5LSJWCB7Y7PORZZHQQ2OR5W2WWTKUTEX2J42NLDVLA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.024842732 (74h30m17.064415852s) [noise=true , address=77.90.31.12:4590 , piece_id=AZGNWLAKPTZK4OQJWFJK452VKSYFWHRXHKDTWMYH37G5S3MJVWZA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025156733 (74h30m17.064729863s) [noise=true , address=77.90.31.21:4878 , piece_id=A7XD5IZ47PHHUML6IP7UC7J7ZBSV55223LLWQHHTIWTGYM2ERXSA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025501453 (74h30m17.065074573s) [noise=true , address=77.90.31.10:4517 , piece_id=SV3AGAFN6VD5PX6UAJDSMFGMPAL6JL4KQY3MLMAVJRLJ3VKAAQQQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.025837175 (74h30m17.065410295s) [noise=true , address=45.91.116.11:4118 , piece_id=55ZVFGBOPFGMEFXCJI2ZLO6JVXT5QIP4MDOUBR3GHGQN2Q5Z2P3A, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026152899 (74h30m17.065726029s) [noise=true , address=45.91.116.7:4009 , piece_id=3UA5Q4YZSM5KEYLBFBVI2VRDESGB7CEGLAVDVMA5NYOUEZACI6XA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026467651 (74h30m17.066040772s) [noise=true , address=45.91.116.6:4975 , piece_id=7PISBWRCIAYWKGKLECRHDDR6R7SJ4RNQCZ4QQMXSYXU4V6E2WU2Q, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.026815467 (74h30m17.066388587s) [noise=true , address=77.90.31.17:4771 , piece_id=SDG7FI2WX2YW6NK6AOI3M3ZF3GMZXOFLRSHQJA4ARAQQHTWSVDRQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027174484 (74h30m17.066747604s) [noise=true , address=77.90.31.21:4879 , piece_id=YTTOURVEYHWVW3KJ3X3S7C5RDMXS7SEQWRHFI7W77672RNELIPEQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027423050 (74h30m17.06699617s) [noise=true , address=45.91.116.14:4189 , piece_id=MEXH26MGR623N44EOKXZAKMR6BPBAXWKWDLNRP5XEDO5QLXHX3SA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.027788729 (74h30m17.067361859s) [noise=true , address=77.90.31.15:4697 , piece_id=OUBA3X3KXDVMTZCTFJUUFUTGE7S5TXD7DUBCFI2KJLFDQCMSQ3QQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028096337 (74h30m17.067669457s) [noise=true , address=45.91.116.8:4048 , piece_id=7S5POEBLCZF6JE53OSANSHNFXVKYMCJR6HX2UIVFM26ZSKU7SFYQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028407233 (74h30m17.067980353s) [noise=true , address=77.90.31.18:4805 , piece_id=2CQVXRFXHMY35FXVZEDTF5E23O4MLQZTS4VVYJ2FWSAFTTJNNOYQ, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:41.028571768 (74h30m17.068144888s) [noise=true , address=45.91.116.5:4297 , piece_id=HAP64MIMSQWN5ROH6HIMCC6B5DTCHG6XKXUBZXAOAFSXCGHFQ2HA, seg_pos={0 0}, upload=65553]: Uploading piece...
Sep 9 15:58:42.880051068 (74h30m18.919624188s) [upload=65553]: Next returned nil segment
Sep 9 15:58:42.880074704 (74h30m18.919647904s) [upload=65553]: Waiting for error group managing segments...
Sep 9 15:58:43.169368198 (74h30m19.208941318s) [noise=true , address=77.90.31.21:4879 , piece_id=YTTOURVEYHWVW3KJ3X3S7C5RDMXS7SEQWRHFI7W77672RNELIPEQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.221767237 (74h30m19.261340357s) [noise=true , address=77.90.31.17:4771 , piece_id=SDG7FI2WX2YW6NK6AOI3M3ZF3GMZXOFLRSHQJA4ARAQQHTWSVDRQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.245871151 (74h30m19.285444261s) [noise=true , address=77.90.31.10:4517 , piece_id=SV3AGAFN6VD5PX6UAJDSMFGMPAL6JL4KQY3MLMAVJRLJ3VKAAQQQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.253603979 (74h30m19.293177099s) [noise=true , address=45.91.116.6:4975 , piece_id=7PISBWRCIAYWKGKLECRHDDR6R7SJ4RNQCZ4QQMXSYXU4V6E2WU2Q, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.267280440 (74h30m19.30685358s) [noise=true , address=45.91.116.7:4009 , piece_id=3UA5Q4YZSM5KEYLBFBVI2VRDESGB7CEGLAVDVMA5NYOUEZACI6XA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.268819054 (74h30m19.308392184s) [noise=true , address=77.90.31.14:4661 , piece_id=LEX3JXQO6JIFJSE3SKQ2XGKQOSWXZSKUPQHQWH5L7U5BKFI6JQ5Q, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.279705877 (74h30m19.319278997s) [noise=true , address=77.90.31.14:4662 , piece_id=WTAAIECILA5LSJWCB7Y7PORZZHQQ2OR5W2WWTKUTEX2J42NLDVLA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.318054870 (74h30m19.35762799s) [noise=true , address=77.90.31.12:4590 , piece_id=AZGNWLAKPTZK4OQJWFJK452VKSYFWHRXHKDTWMYH37G5S3MJVWZA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.355173029 (74h30m19.394746149s) [noise=true , address=77.90.31.21:4878 , piece_id=A7XD5IZ47PHHUML6IP7UC7J7ZBSV55223LLWQHHTIWTGYM2ERXSA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.521692446 (74h30m19.561265576s) [noise=true , address=45.91.116.5:4297 , piece_id=HAP64MIMSQWN5ROH6HIMCC6B5DTCHG6XKXUBZXAOAFSXCGHFQ2HA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.550762924 (74h30m19.590336044s) [noise=true , address=77.90.31.18:4805 , piece_id=2CQVXRFXHMY35FXVZEDTF5E23O4MLQZTS4VVYJ2FWSAFTTJNNOYQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.617186440 (74h30m19.65675957s) [noise=true , address=45.91.116.14:4189 , piece_id=MEXH26MGR623N44EOKXZAKMR6BPBAXWKWDLNRP5XEDO5QLXHX3SA, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.628619508 (74h30m19.668192629s) [noise=true , address=77.90.31.15:4697 , piece_id=OUBA3X3KXDVMTZCTFJUUFUTGE7S5TXD7DUBCFI2KJLFDQCMSQ3QQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: <nil>
Sep 9 15:58:43.628645568 (74h30m19.668218688s) [seg_pos={0 0}, upload=65553]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:58:43.628879677 (74h30m19.668452797s) [noise=true , address=45.91.116.11:4118 , piece_id=55ZVFGBOPFGMEFXCJI2ZLO6JVXT5QIP4MDOUBR3GHGQN2Q5Z2P3A, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56308->45.91.116.11:4118: use of closed network connection
Sep 9 15:58:43.628921888 (74h30m19.668495008s) [noise=true , address=45.91.116.11:4117 , piece_id=SF6577AWKAHTDELU4P7DNNUG4FJR76GYQXQ4NBB2PMDISNUE7NWQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection; write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40370->45.91.116.11:4117: use of closed network connection
Sep 9 15:58:43.628979578 (74h30m19.668552698s) [noise=true , address=45.91.116.8:4048 , piece_id=7S5POEBLCZF6JE53OSANSHNFXVKYMCJR6HX2UIVFM26ZSKU7SFYQ, seg_pos={0 0}, upload=65553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF
Sep 9 15:58:43.629039723 (74h30m19.668612843s) [seg_pos={0 0}, upload=65553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:58:43.629138882 (74h30m19.668712012s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:58:43.629144594 (74h30m19.668717714s) [upload=65553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:58:43.968648710 (74h30m20.00822184s) [upload=65553]: Done uploading segments...
Sep 9 15:58:43.968671363 (74h30m20.008244533s) [upload=65553]: Upload finished. err: <nil>
Duration: 30.00269202s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
709.336728ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:58:49 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 709.361145ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X.X....X...................X...............................] duration:
7.015736358s
FAIL (2025-09-09 15:58:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.01575859s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.615443052s
FAIL (2025-09-09 15:58:30 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.615465366s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.707553262s
FAIL (2025-09-09 15:58:13 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.707574754s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X.X....X...................X..............................] duration:
6.595761861s
FAIL (2025-09-09 15:58:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.595784124s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
53 out of 60 is failed (threshold=3) [..XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.002174678s
FAIL (2025-09-09 15:57:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 15:57:28.687726826 (74h29m4.727299946s) [upload=65535]: Starting upload
Sep 9 15:57:28.687791770 (74h29m4.72736489s) [upload=65535]: Done starting upload
Sep 9 15:57:28.687850152 (74h29m4.727423272s) [upload=65535]: Uploading segments...
Sep 9 15:57:28.687996832 (74h29m4.727569952s) [upload=65535]: Got next segment. Inline: false
Sep 9 15:57:28.688041638 (74h29m4.727614758s) [upload=65535]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:57:28.688046718 (74h29m4.727619838s) [upload=65535]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:57:28.945441161 (74h29m4.985014281s) [seg_pos={0 0}, upload=65535]: Begin upload segment...
Sep 9 15:57:28.945505895 (74h29m4.985079015s) [seg_pos={0 0}, upload=65535]: Done begin upload segment.
Sep 9 15:57:28.945530081 (74h29m4.985103191s) [noise=true , address=145.79.227.18:4007 , piece_id=HNUGEPHY2TIJ66CS6MTMXQJP7KZJEJKRH3A4NUQYXHD5JXFRTP7Q, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.945564518 (74h29m4.985137638s) [noise=true , address=77.90.31.10:4518 , piece_id=A77BNM4GCPTANPCHKILCO6OTVN6ODLIAECK4TBHZTBWWCLSBG22A, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.946023586 (74h29m4.985596696s) [noise=true , address=45.91.116.6:4973 , piece_id=BZA3YOMON5GFBM6OX76MBRIFB4SF7GGJYNFZUAWQMGCFJF6VNKDQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.946309273 (74h29m4.985882393s) [noise=true , address=45.91.116.5:4298 , piece_id=LDU3QIHBUCKPPNR3UZEYKSFZXG74TCGFEHPI2HCUR3KOOVPN64GQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.946584870 (74h29m4.98615798s) [noise=true , address=77.90.31.12:4592 , piece_id=TCSZE7TM4ARG7WMA4I4RRMVL4F5GPYGDD6CJEZ2YFHP6VNN5UTJQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.946760566 (74h29m4.986333686s) [noise=true , address=45.91.116.14:4189 , piece_id=ZIF4WKR4CFZXKDXE5M2LZMKDQXB2CHETKPYBLFXGZV2CF2A4EKRQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.946850368 (74h29m4.986423478s) [noise=true , address=45.91.116.9:4262 , piece_id=XIZ2PJPZ5FYL3NQIL35XMWMZPB75U77GSR63EPCCTTM6TV5KPO4Q, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.947092552 (74h29m4.986665672s) [noise=true , address=45.91.116.8:4045 , piece_id=T4UAGAD5HKWJW7CCCVLKQRNPDC7YMIFKTTYEHLCWRPMQE2DW2N3Q, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.947221758 (74h29m4.986794878s) [noise=true , address=77.90.31.13:4627 , piece_id=3HWC7DDZL6QSKWWKOX5OYZLZM5K7IG2QOINLFGJWCCONM2ERQOJQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.947329004 (74h29m4.986902114s) [noise=true , address=77.90.31.19:4843 , piece_id=7KFGHEBPGG2N63KI6BNZOCM5IPORIVQOYWXMQ4DAMD42HMCDYTCQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.947651541 (74h29m4.987224661s) [noise=true , address=45.91.116.14:4190 , piece_id=X4DQM5RMSD3JYO6KZRG2LWKSXZIYBMUMV66CV4IG36MXLK4RS6PA, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.947688522 (74h29m4.987261642s) [noise=true , address=77.90.31.14:4663 , piece_id=D537YSYXWYAUN2FKYPB6UIVIMS252JWX7SSAVUIBQ42MO5WYAHVQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.948086103 (74h29m4.987659233s) [noise=true , address=45.91.116.13:4155 , piece_id=WPLHVBYGWHHITICGGVNBJGFBW4OTN7G72IJH4ZV7RXTIZPVENZRQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.948453627 (74h29m4.988026747s) [noise=true , address=145.79.227.3:4149 , piece_id=A6OBKXI7ZIA62H52LHGSLZA7OWC7ZF6ZNZ6EM73WTQTQRQZ22QMQ, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.950308726 (74h29m4.989881847s) [noise=true , address=45.91.116.12:4227 , piece_id=5QY5RZ4YG5V2ZOR5M6ZH64SAOAPUHJHYPLLHSA4L5B2562XBZE2Q, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:28.951325301 (74h29m4.990898422s) [noise=true , address=45.91.116.4:4939 , piece_id=BKWOA2HFFVSJWRWQ3KSSB4QJWMMTK7V2IVQDN5BMDAPUPDR4USWA, seg_pos={0 0}, upload=65535]: Uploading piece...
Sep 9 15:57:30.745231645 (74h29m6.784804775s) [upload=65535]: Next returned nil segment
Sep 9 15:57:30.745251282 (74h29m6.784824413s) [upload=65535]: Waiting for error group managing segments...
Sep 9 15:57:31.088717059 (74h29m7.128290179s) [noise=true , address=77.90.31.12:4592 , piece_id=TCSZE7TM4ARG7WMA4I4RRMVL4F5GPYGDD6CJEZ2YFHP6VNN5UTJQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.159349900 (74h29m7.19892302s) [noise=true , address=45.91.116.14:4190 , piece_id=X4DQM5RMSD3JYO6KZRG2LWKSXZIYBMUMV66CV4IG36MXLK4RS6PA, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.161688866 (74h29m7.201261986s) [noise=true , address=45.91.116.4:4939 , piece_id=BKWOA2HFFVSJWRWQ3KSSB4QJWMMTK7V2IVQDN5BMDAPUPDR4USWA, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.167709287 (74h29m7.207282407s) [noise=true , address=45.91.116.14:4189 , piece_id=ZIF4WKR4CFZXKDXE5M2LZMKDQXB2CHETKPYBLFXGZV2CF2A4EKRQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.186565620 (74h29m7.22613874s) [noise=true , address=77.90.31.13:4627 , piece_id=3HWC7DDZL6QSKWWKOX5OYZLZM5K7IG2QOINLFGJWCCONM2ERQOJQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.204876579 (74h29m7.244449699s) [noise=true , address=77.90.31.19:4843 , piece_id=7KFGHEBPGG2N63KI6BNZOCM5IPORIVQOYWXMQ4DAMD42HMCDYTCQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.228445990 (74h29m7.268019111s) [noise=true , address=77.90.31.10:4518 , piece_id=A77BNM4GCPTANPCHKILCO6OTVN6ODLIAECK4TBHZTBWWCLSBG22A, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.234888058 (74h29m7.274461178s) [noise=true , address=45.91.116.6:4973 , piece_id=BZA3YOMON5GFBM6OX76MBRIFB4SF7GGJYNFZUAWQMGCFJF6VNKDQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.260059886 (74h29m7.299633006s) [noise=true , address=45.91.116.8:4045 , piece_id=T4UAGAD5HKWJW7CCCVLKQRNPDC7YMIFKTTYEHLCWRPMQE2DW2N3Q, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.263862042 (74h29m7.303435162s) [noise=true , address=45.91.116.5:4298 , piece_id=LDU3QIHBUCKPPNR3UZEYKSFZXG74TCGFEHPI2HCUR3KOOVPN64GQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.293385427 (74h29m7.332958557s) [noise=true , address=45.91.116.12:4227 , piece_id=5QY5RZ4YG5V2ZOR5M6ZH64SAOAPUHJHYPLLHSA4L5B2562XBZE2Q, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.305783582 (74h29m7.345356702s) [noise=true , address=77.90.31.14:4663 , piece_id=D537YSYXWYAUN2FKYPB6UIVIMS252JWX7SSAVUIBQ42MO5WYAHVQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.461920331 (74h29m7.501493451s) [noise=true , address=145.79.227.3:4149 , piece_id=A6OBKXI7ZIA62H52LHGSLZA7OWC7ZF6ZNZ6EM73WTQTQRQZ22QMQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: <nil>
Sep 9 15:57:31.461939718 (74h29m7.501512838s) [seg_pos={0 0}, upload=65535]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:57:31.463412657 (74h29m7.502985777s) [noise=true , address=45.91.116.9:4262 , piece_id=XIZ2PJPZ5FYL3NQIL35XMWMZPB75U77GSR63EPCCTTM6TV5KPO4Q, seg_pos={0 0}, upload=65535]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:39258->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:39258->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39258->45.91.116.9:4262: use of closed network connection
Sep 9 15:57:31.463459996 (74h29m7.503033117s) [noise=true , address=45.91.116.13:4155 , piece_id=WPLHVBYGWHHITICGGVNBJGFBW4OTN7G72IJH4ZV7RXTIZPVENZRQ, seg_pos={0 0}, upload=65535]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:38038->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:38038->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38038->45.91.116.13:4155: use of closed network connection
Sep 9 15:57:31.463497479 (74h29m7.503070589s) [noise=true , address=145.79.227.18:4007 , piece_id=HNUGEPHY2TIJ66CS6MTMXQJP7KZJEJKRH3A4NUQYXHD5JXFRTP7Q, seg_pos={0 0}, upload=65535]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 188.166.225.170:46856->145.79.227.18:4007: use of closed network connection; write tcp 188.166.225.170:46856->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46856->145.79.227.18:4007: use of closed network connection
Sep 9 15:57:31.463508059 (74h29m7.503081179s) [seg_pos={0 0}, upload=65535]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:57:31.463597319 (74h29m7.503170439s) [upload=65535]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:57:31.463601358 (74h29m7.503174478s) [upload=65535]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:57:31.845931723 (74h29m7.885504844s) [upload=65535]: Done uploading segments...
Sep 9 15:57:31.845951340 (74h29m7.88552446s) [upload=65535]: Upload finished. err: <nil>
Duration: 30.002190197s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...X.X....X...................X.............................] duration:
6.910772689s
FAIL (2025-09-09 15:57:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.910794992s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
711.494106ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:57:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 711.51737ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.536539896s
FAIL (2025-09-09 15:57:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.536561777s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
703.218621ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:57:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 703.245522ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
782.697881ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:57:01 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 782.72882ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
53 out of 60 is failed (threshold=3) [X..XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.0021384s
FAIL (2025-09-09 15:56:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 15:56:30.178327928 (74h28m6.217901048s) [upload=65523]: Starting upload
Sep 9 15:56:30.178404435 (74h28m6.217977555s) [upload=65523]: Done starting upload
Sep 9 15:56:30.178510308 (74h28m6.218083428s) [upload=65523]: Uploading segments...
Sep 9 15:56:30.178601752 (74h28m6.218174872s) [upload=65523]: Got next segment. Inline: false
Sep 9 15:56:30.178634525 (74h28m6.218207645s) [upload=65523]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:56:30.178638612 (74h28m6.218211732s) [upload=65523]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:56:30.708651583 (74h28m6.748224703s) [seg_pos={0 0}, upload=65523]: Begin upload segment...
Sep 9 15:56:30.708694485 (74h28m6.748267605s) [seg_pos={0 0}, upload=65523]: Done begin upload segment.
Sep 9 15:56:30.708723209 (74h28m6.748296329s) [noise=true , address=145.79.227.16:4910 , piece_id=O5TWR63KABKERG574HU7NK7KWAWROERDG3S2J5X2GBS4QHCPKJ4A, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.709256370 (74h28m6.74882949s) [noise=true , address=45.91.116.5:4298 , piece_id=NPRBIQURPN3ENEGXIN5QOTVN7KI4GZTZIACKOA2W5NZAWIKJ7FCQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.709540824 (74h28m6.749113944s) [noise=true , address=77.90.31.18:4807 , piece_id=KYHKSGXWPKRA67DGFLS4KDPP6XX5RREXJWTXYFOFOJOVXRS2QH6A, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.709830419 (74h28m6.749403539s) [noise=true , address=45.91.116.6:4974 , piece_id=2E3J63Q5KGRHJOJGJ5IULOJWNLJ6A3QTUKVTLKABAMLOBXRJIPYQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.710941935 (74h28m6.750515055s) [noise=true , address=45.91.116.8:4048 , piece_id=5WUWLKC7DH2XZCF6FRQ5U5BMCR7BCSZUMSRKLFKSX3KLLSJA54JQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.711205810 (74h28m6.75077893s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EBU3SWJYJYDLHX74G7N2HU73NQ4XZDP7TUSTIKKLW7S2RIL4GQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.711418808 (74h28m6.750991928s) [noise=true , address=77.90.31.16:4735 , piece_id=UCMQP7PDOSDE75C4HEBIJ5PQXIYHPFQUT4T37VXB3CUYILFTXCVA, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.711695557 (74h28m6.751268677s) [noise=true , address=45.91.116.6:4973 , piece_id=GTWVWNQQXIN6G6NHPXJQZQVK36G2A2D2GSUKQBECK2VKQUR4B74Q, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.711899657 (74h28m6.751472777s) [noise=true , address=77.90.31.19:4842 , piece_id=FX3TFHWV3A44KZNKEEACONFFOV7IQKUXISI5MU27BLR7U26M7FUQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.712101604 (74h28m6.751674725s) [noise=true , address=77.90.31.10:4519 , piece_id=N5KFN2NLINWT4K27L6GIOGXSET6I7XXLANIEF2DFAXFSV4QG6MTA, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.712303510 (74h28m6.75187663s) [noise=true , address=45.91.116.9:4263 , piece_id=U2XQGP5PIAU7OKKAE2KOWHYMOXB7JXVOZXATOBWP4I3YZEBTBPAQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.712503674 (74h28m6.752076794s) [noise=true , address=45.91.116.4:4940 , piece_id=WWCY3WY2A33HUMSMPF6WFMM424Q6XTKRB7LWMDZGQW3OMCKAP47Q, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.713312621 (74h28m6.752885742s) [noise=true , address=45.91.116.12:4225 , piece_id=ZWEFBACEJ2LCZ4WJ5T5F6JPI5CCWK6SQGBI4ERA4ID3ZVWYP7YBA, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.713631362 (74h28m6.753204482s) [noise=true , address=45.91.116.10:4082 , piece_id=WEE77RU2AO7YVEOPS5WIKJUSTR5IXOD3FRMG4JCMVD3UYQKSBCRQ, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.713864497 (74h28m6.753437617s) [noise=true , address=45.91.116.13:4153 , piece_id=B75GSRIF73NBKNI3TCL736YXOFNJPYKVFDKUQ6ZWZV5RC6GVLAEA, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:30.714142469 (74h28m6.753715589s) [noise=true , address=45.91.116.4:4938 , piece_id=KET6WAJBTSMD56BQ43IMMGFKTAXAEQZWFH2EAEI5BDGD7QYINWIA, seg_pos={0 0}, upload=65523]: Uploading piece...
Sep 9 15:56:32.609446773 (74h28m8.649019893s) [upload=65523]: Next returned nil segment
Sep 9 15:56:32.609464938 (74h28m8.649038058s) [upload=65523]: Waiting for error group managing segments...
Sep 9 15:56:32.917752879 (74h28m8.957326009s) [noise=true , address=77.90.31.18:4807 , piece_id=KYHKSGXWPKRA67DGFLS4KDPP6XX5RREXJWTXYFOFOJOVXRS2QH6A, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.941906580 (74h28m8.9814797s) [noise=true , address=45.91.116.4:4940 , piece_id=WWCY3WY2A33HUMSMPF6WFMM424Q6XTKRB7LWMDZGQW3OMCKAP47Q, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.943431507 (74h28m8.983004617s) [noise=true , address=45.91.116.6:4974 , piece_id=2E3J63Q5KGRHJOJGJ5IULOJWNLJ6A3QTUKVTLKABAMLOBXRJIPYQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.958152938 (74h28m8.997726068s) [noise=true , address=45.91.116.10:4082 , piece_id=WEE77RU2AO7YVEOPS5WIKJUSTR5IXOD3FRMG4JCMVD3UYQKSBCRQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.964355066 (74h28m9.003928186s) [noise=true , address=45.91.116.4:4938 , piece_id=KET6WAJBTSMD56BQ43IMMGFKTAXAEQZWFH2EAEI5BDGD7QYINWIA, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.978524721 (74h28m9.018097841s) [noise=true , address=77.90.31.16:4735 , piece_id=UCMQP7PDOSDE75C4HEBIJ5PQXIYHPFQUT4T37VXB3CUYILFTXCVA, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.982691585 (74h28m9.022264705s) [noise=true , address=145.79.227.16:4910 , piece_id=O5TWR63KABKERG574HU7NK7KWAWROERDG3S2J5X2GBS4QHCPKJ4A, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:32.998683526 (74h28m9.038256647s) [noise=true , address=77.90.31.19:4842 , piece_id=FX3TFHWV3A44KZNKEEACONFFOV7IQKUXISI5MU27BLR7U26M7FUQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.057531576 (74h28m9.097104706s) [noise=true , address=45.91.116.6:4973 , piece_id=GTWVWNQQXIN6G6NHPXJQZQVK36G2A2D2GSUKQBECK2VKQUR4B74Q, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.123184168 (74h28m9.162757288s) [noise=true , address=45.91.116.12:4225 , piece_id=ZWEFBACEJ2LCZ4WJ5T5F6JPI5CCWK6SQGBI4ERA4ID3ZVWYP7YBA, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.268263919 (74h28m9.307837039s) [noise=true , address=45.91.116.13:4153 , piece_id=B75GSRIF73NBKNI3TCL736YXOFNJPYKVFDKUQ6ZWZV5RC6GVLAEA, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.285429146 (74h28m9.325002266s) [noise=true , address=45.91.116.8:4048 , piece_id=5WUWLKC7DH2XZCF6FRQ5U5BMCR7BCSZUMSRKLFKSX3KLLSJA54JQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.314693826 (74h28m9.354266936s) [noise=true , address=45.91.116.5:4298 , piece_id=NPRBIQURPN3ENEGXIN5QOTVN7KI4GZTZIACKOA2W5NZAWIKJ7FCQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: <nil>
Sep 9 15:56:33.314718423 (74h28m9.354291543s) [seg_pos={0 0}, upload=65523]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:56:33.314845527 (74h28m9.354418647s) [noise=true , address=45.91.116.9:4263 , piece_id=U2XQGP5PIAU7OKKAE2KOWHYMOXB7JXVOZXATOBWP4I3YZEBTBPAQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF
Sep 9 15:56:33.314992017 (74h28m9.354565137s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EBU3SWJYJYDLHX74G7N2HU73NQ4XZDP7TUSTIKKLW7S2RIL4GQ, seg_pos={0 0}, upload=65523]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: expected piece hash; context canceled; EOF
Sep 9 15:56:33.315031082 (74h28m9.354604202s) [noise=true , address=77.90.31.10:4519 , piece_id=N5KFN2NLINWT4K27L6GIOGXSET6I7XXLANIEF2DFAXFSV4QG6MTA, seg_pos={0 0}, upload=65523]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:48378->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:48378->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48378->77.90.31.10:4519: use of closed network connection
Sep 9 15:56:33.315044798 (74h28m9.354617918s) [seg_pos={0 0}, upload=65523]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:56:33.315165099 (74h28m9.354738219s) [upload=65523]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:56:33.315198532 (74h28m9.354771652s) [upload=65523]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:56:33.681351522 (74h28m9.720924652s) [upload=65523]: Done uploading segments...
Sep 9 15:56:33.681373023 (74h28m9.720946144s) [upload=65523]: Upload finished. err: <nil>
Duration: 30.002161324s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X.X....X...................X............................] duration:
6.92131512s
FAIL (2025-09-09 15:56:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.921330689s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
721.586019ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:56:36 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 721.611326ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.685939103s
FAIL (2025-09-09 15:56:33 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.685998377s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
790.412653ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:56:19 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 790.434314ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
53 out of 60 is failed (threshold=3) [XX..XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.00216045s
FAIL (2025-09-09 15:56:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 15:55:37.312881627 (74h27m13.352454758s) [upload=65508]: Starting upload
Sep 9 15:55:37.312960719 (74h27m13.352533839s) [upload=65508]: Done starting upload
Sep 9 15:55:37.313082742 (74h27m13.352655872s) [upload=65508]: Uploading segments...
Sep 9 15:55:37.313214484 (74h27m13.352787604s) [upload=65508]: Got next segment. Inline: false
Sep 9 15:55:37.313247948 (74h27m13.352821068s) [upload=65508]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:55:37.313252376 (74h27m13.352825486s) [upload=65508]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:55:37.611324190 (74h27m13.65089731s) [seg_pos={0 0}, upload=65508]: Begin upload segment...
Sep 9 15:55:37.611367022 (74h27m13.650940142s) [seg_pos={0 0}, upload=65508]: Done begin upload segment.
Sep 9 15:55:37.611398051 (74h27m13.650971171s) [noise=true , address=145.79.227.14:4840 , piece_id=IOGGTWR55QB3XA7RPFYLIVOBG43INEYSFM42F3OQM3AZNUPWRSEA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.611409232 (74h27m13.650982352s) [noise=true , address=77.90.31.13:4627 , piece_id=WRGAUO4U7HZRETQPK27TUUIL3N6Q7DY6ADTB6M4S4QZPXVICARUA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.611902967 (74h27m13.651476087s) [noise=true , address=45.91.116.12:4226 , piece_id=POBIKXCVGKLJ3CIQ2EQXGBF2P4TJC4PV5ZP7F7GN2QGRMM2BFJRA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.611975937 (74h27m13.651549057s) [noise=true , address=45.91.116.5:4299 , piece_id=SMV2C3QJ2WSTJFT2SNVPPTXBCOLBLBL4OOAMXUXE6QLEUPEFFW6A, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.612181269 (74h27m13.651754389s) [noise=true , address=45.91.116.12:4227 , piece_id=HYEFHPXQYSEMKZYJDXQAQBJY4NKEK7IYFPHW3B3NKCGQRM6OK4KQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.612308914 (74h27m13.651882064s) [noise=true , address=45.91.116.14:4190 , piece_id=FQZQGRMJR55CVVDBVPH6XW3BO73YSSFCAA5KGYPJTS7R4DJ3H2MQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.612447499 (74h27m13.652020619s) [noise=true , address=145.79.227.16:4909 , piece_id=QE5U42LWGJNKFWQYIHA3WEPLET6QF3XJZXFSZND7FNJSMHVW7JPA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.612676527 (74h27m13.652249647s) [noise=true , address=77.90.31.14:4661 , piece_id=CZ4OR55THZCE3BU7PUA64S57CULHKLLTXFZBWCPANXCQ7OQK2LNQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.612758765 (74h27m13.652331895s) [noise=true , address=77.90.31.10:4517 , piece_id=2LMEVQTVJ2J6RVCAHHLZF5RQYPT3K7KKWPATEBVVWIEOJAJTLZIA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.613001399 (74h27m13.652574519s) [noise=true , address=45.91.116.7:4010 , piece_id=ZGS7PURXIIXQ7RC2G2RBWPULTJPKBABV2L53ZYOI7Q7BXEXE5H2A, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.613130295 (74h27m13.652703416s) [noise=true , address=45.91.116.4:4939 , piece_id=DA3PN2ECJ2OP74B6I5YU5XJWOZMBHMLPQTK2NYOANMRVMID63Y3A, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.613299950 (74h27m13.65287307s) [noise=true , address=77.90.31.16:4733 , piece_id=3UHI6UDNXYNMUPQPWKLXCXSD5S3OH56HWF52CLMOFK7LAOIMVMXQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.613513328 (74h27m13.653086448s) [noise=true , address=45.91.116.5:4298 , piece_id=F5VPTHUFQ5DJ4MJ6NV5WFGZZ6SZCDJ5FQM4PKLPR2ATMB3ALHPYQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.613601116 (74h27m13.653174236s) [noise=true , address=77.90.31.17:4771 , piece_id=ZBYC5XFY4EWVVRJRAKAH7PGWR5H5JNE7F2PRRXGNKRQ2PX3ZY4NA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.615228531 (74h27m13.654801651s) [noise=true , address=45.91.116.11:4119 , piece_id=G47DCZI5ENH6ANAH2TESVINQG5L3RFGBVULJ4UMNQPXZJXE7WXBQ, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:37.615350966 (74h27m13.654924086s) [noise=true , address=45.91.116.13:4154 , piece_id=FLE6O6KRLULGG2Y75CDEC6LKXDN25FLZWKFK2X4CP7K4FPCW5ERA, seg_pos={0 0}, upload=65508]: Uploading piece...
Sep 9 15:55:39.529987393 (74h27m15.569560513s) [upload=65508]: Next returned nil segment
Sep 9 15:55:39.530008163 (74h27m15.569581283s) [upload=65508]: Waiting for error group managing segments...
Sep 9 15:55:39.834297513 (74h27m15.873870633s) [noise=true , address=45.91.116.7:4010 , piece_id=ZGS7PURXIIXQ7RC2G2RBWPULTJPKBABV2L53ZYOI7Q7BXEXE5H2A, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.842525568 (74h27m15.882098698s) [noise=true , address=45.91.116.12:4227 , piece_id=HYEFHPXQYSEMKZYJDXQAQBJY4NKEK7IYFPHW3B3NKCGQRM6OK4KQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.853820120 (74h27m15.89339324s) [noise=true , address=45.91.116.11:4119 , piece_id=G47DCZI5ENH6ANAH2TESVINQG5L3RFGBVULJ4UMNQPXZJXE7WXBQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.873065038 (74h27m15.912638158s) [noise=true , address=77.90.31.10:4517 , piece_id=2LMEVQTVJ2J6RVCAHHLZF5RQYPT3K7KKWPATEBVVWIEOJAJTLZIA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.875846599 (74h27m15.915419719s) [noise=true , address=45.91.116.14:4190 , piece_id=FQZQGRMJR55CVVDBVPH6XW3BO73YSSFCAA5KGYPJTS7R4DJ3H2MQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.881735268 (74h27m15.921308388s) [noise=true , address=45.91.116.5:4298 , piece_id=F5VPTHUFQ5DJ4MJ6NV5WFGZZ6SZCDJ5FQM4PKLPR2ATMB3ALHPYQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.901745861 (74h27m15.941318981s) [noise=true , address=77.90.31.17:4771 , piece_id=ZBYC5XFY4EWVVRJRAKAH7PGWR5H5JNE7F2PRRXGNKRQ2PX3ZY4NA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.904935674 (74h27m15.944508794s) [noise=true , address=77.90.31.14:4661 , piece_id=CZ4OR55THZCE3BU7PUA64S57CULHKLLTXFZBWCPANXCQ7OQK2LNQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.908792244 (74h27m15.948365354s) [noise=true , address=45.91.116.12:4226 , piece_id=POBIKXCVGKLJ3CIQ2EQXGBF2P4TJC4PV5ZP7F7GN2QGRMM2BFJRA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.932818880 (74h27m15.972392011s) [noise=true , address=45.91.116.5:4299 , piece_id=SMV2C3QJ2WSTJFT2SNVPPTXBCOLBLBL4OOAMXUXE6QLEUPEFFW6A, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.940719710 (74h27m15.98029284s) [noise=true , address=45.91.116.13:4154 , piece_id=FLE6O6KRLULGG2Y75CDEC6LKXDN25FLZWKFK2X4CP7K4FPCW5ERA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.953259074 (74h27m15.992832194s) [noise=true , address=77.90.31.16:4733 , piece_id=3UHI6UDNXYNMUPQPWKLXCXSD5S3OH56HWF52CLMOFK7LAOIMVMXQ, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.957039930 (74h27m15.99661305s) [noise=true , address=45.91.116.4:4939 , piece_id=DA3PN2ECJ2OP74B6I5YU5XJWOZMBHMLPQTK2NYOANMRVMID63Y3A, seg_pos={0 0}, upload=65508]: Done uploading piece. err: <nil>
Sep 9 15:55:39.957055309 (74h27m15.996628429s) [seg_pos={0 0}, upload=65508]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:55:39.957148517 (74h27m15.996721638s) [noise=true , address=145.79.227.16:4909 , piece_id=QE5U42LWGJNKFWQYIHA3WEPLET6QF3XJZXFSZND7FNJSMHVW7JPA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Sep 9 15:55:39.957225825 (74h27m15.996798945s) [noise=true , address=77.90.31.13:4627 , piece_id=WRGAUO4U7HZRETQPK27TUUIL3N6Q7DY6ADTB6M4S4QZPXVICARUA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF
Sep 9 15:55:39.957282222 (74h27m15.996855342s) [noise=true , address=145.79.227.14:4840 , piece_id=IOGGTWR55QB3XA7RPFYLIVOBG43INEYSFM42F3OQM3AZNUPWRSEA, seg_pos={0 0}, upload=65508]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF
Sep 9 15:55:39.957295158 (74h27m15.996868278s) [seg_pos={0 0}, upload=65508]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:55:39.957348249 (74h27m15.996921369s) [upload=65508]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:55:39.957354882 (74h27m15.996928002s) [upload=65508]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:55:40.263825516 (74h27m16.303398636s) [upload=65508]: Done uploading segments...
Sep 9 15:55:40.263844462 (74h27m16.303417582s) [upload=65508]: Upload finished. err: <nil>
Duration: 30.002184777s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....X.X....X...................X...........................] duration:
7.032578787s
FAIL (2025-09-09 15:55:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.032600208s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.518095841s
FAIL (2025-09-09 15:55:40 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.51811628s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
688.577955ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:55:09 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 688.614014ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
52 out of 60 is failed (threshold=3) [.XX..XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001413888s
FAIL (2025-09-09 15:54:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 15:54:28.548754855 (74h26m4.588327975s) [upload=65495]: Starting upload
Sep 9 15:54:28.548809940 (74h26m4.58838306s) [upload=65495]: Done starting upload
Sep 9 15:54:28.548867831 (74h26m4.588440951s) [upload=65495]: Uploading segments...
Sep 9 15:54:28.548981117 (74h26m4.588554237s) [upload=65495]: Got next segment. Inline: false
Sep 9 15:54:28.549010063 (74h26m4.588583183s) [upload=65495]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:54:28.549014371 (74h26m4.588587491s) [upload=65495]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:54:28.792891448 (74h26m4.832464568s) [seg_pos={0 0}, upload=65495]: Begin upload segment...
Sep 9 15:54:28.792932867 (74h26m4.832505987s) [seg_pos={0 0}, upload=65495]: Done begin upload segment.
Sep 9 15:54:28.792963345 (74h26m4.832536465s) [noise=true , address=45.91.116.11:4117 , piece_id=OGSGTNDG3PMOLGWFTTYAMI4EI2MFDE23L3OOBREZDIX6EBS4AVOA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.792978815 (74h26m4.832551925s) [noise=true , address=45.91.116.5:4298 , piece_id=NQ3IIUXCWW2JBODUZAVSTTCU3AI4LBXEGSV7UWSGZK4GSACTNQSA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793019703 (74h26m4.832592823s) [noise=true , address=145.79.227.7:4294 , piece_id=OR2GKH5WITNRCE7IHE6KJZHDLU6FWHKQR5GXSKTCMPWGFIN5E6UQ, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793349875 (74h26m4.832922995s) [noise=true , address=77.90.31.18:4807 , piece_id=IOM3PNGOFK4EARUW3HW62XEHB5CQ7J55UQL3B6EBWF7MRRRJ242Q, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793358963 (74h26m4.832932083s) [noise=true , address=45.91.116.7:4010 , piece_id=OI5G7L63KPHDHQZ43E56YKR4WEGBOBRIR42VLVHNZBW3E63N75AQ, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793628779 (74h26m4.833201899s) [noise=true , address=45.91.116.4:4937 , piece_id=E56MCYNPR6NMSX2ZQ7ERY6OCWRHDHSZAM4WLMRM6WKXUOZ5XEMPA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793642825 (74h26m4.833215945s) [noise=true , address=77.90.31.18:4806 , piece_id=O6MX7VK646TNIN6D6MFVXVUSZ7W3UK63JAK53DKV6M7H5FBWPQ7A, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793929454 (74h26m4.833502574s) [noise=true , address=45.91.116.13:4153 , piece_id=KI5HR3UCDG7LL2ZSHZSNFBCDLW65HNK5HOXT3ORXQEH5WPRADICA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.793957348 (74h26m4.833530468s) [noise=true , address=77.90.31.17:4770 , piece_id=IF4XNFYZXLUUZZHRGNUZOKM7AJVWJ6RDOIEGRIW4YGXKTHPDQROQ, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794213898 (74h26m4.833787018s) [noise=true , address=45.91.116.10:4083 , piece_id=MGM4SPIZGII4DPBRNJ335LX372ORMFUGIF2NI55BNQ5FNNWSWINA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794219128 (74h26m4.833792248s) [noise=true , address=45.91.116.4:4939 , piece_id=KIEAXZP5NLYZHY6WYEKH7JEKMBL2LYZ2T6XJPCL6NAU77AIXE6EA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794241752 (74h26m4.833814862s) [noise=true , address=77.90.31.21:4877 , piece_id=UVQR4D57DFTWW2WEHMSOXJ6E7LRYIMPRDHNSIBVGW4L7T2FJJC3A, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794452565 (74h26m4.834025685s) [noise=true , address=45.91.116.14:4189 , piece_id=ZA2F6H4WROEPZOWJPVWZA374IZDUH4SF7PPUTXPIT76RBV2ICZFA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794460079 (74h26m4.834033199s) [noise=true , address=45.91.116.12:4226 , piece_id=EOMY3COIFL64GAGFPLXXZ2YX3GAJXRMEKTMOKZGA4EVIJMR5BS2A, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794701411 (74h26m4.834274531s) [noise=true , address=77.90.31.15:4699 , piece_id=UQ4OYU7ZRNG2P4HPOFJH3VOQX4ZPN5XUUIT34CPQAASKTWGEGBOA, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:28.794936041 (74h26m4.834509161s) [noise=true , address=45.91.116.6:4973 , piece_id=E3V6FUPMCI67SNXVRF6OQILEY7S737NF5FDPQDECBA2M6YDRXP2Q, seg_pos={0 0}, upload=65495]: Uploading piece...
Sep 9 15:54:30.681067782 (74h26m6.720640903s) [upload=65495]: Next returned nil segment
Sep 9 15:54:30.681086498 (74h26m6.720659608s) [upload=65495]: Waiting for error group managing segments...
Sep 9 15:54:30.983046050 (74h26m7.02261917s) [noise=true , address=45.91.116.14:4189 , piece_id=ZA2F6H4WROEPZOWJPVWZA374IZDUH4SF7PPUTXPIT76RBV2ICZFA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.067018437 (74h26m7.106591557s) [noise=true , address=45.91.116.11:4117 , piece_id=OGSGTNDG3PMOLGWFTTYAMI4EI2MFDE23L3OOBREZDIX6EBS4AVOA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.085485505 (74h26m7.125058625s) [noise=true , address=45.91.116.12:4226 , piece_id=EOMY3COIFL64GAGFPLXXZ2YX3GAJXRMEKTMOKZGA4EVIJMR5BS2A, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.086171487 (74h26m7.125744607s) [noise=true , address=45.91.116.7:4010 , piece_id=OI5G7L63KPHDHQZ43E56YKR4WEGBOBRIR42VLVHNZBW3E63N75AQ, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.089008977 (74h26m7.128582137s) [noise=true , address=77.90.31.18:4807 , piece_id=IOM3PNGOFK4EARUW3HW62XEHB5CQ7J55UQL3B6EBWF7MRRRJ242Q, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.107047996 (74h26m7.146621116s) [noise=true , address=77.90.31.21:4877 , piece_id=UVQR4D57DFTWW2WEHMSOXJ6E7LRYIMPRDHNSIBVGW4L7T2FJJC3A, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.174034049 (74h26m7.213607169s) [noise=true , address=45.91.116.13:4153 , piece_id=KI5HR3UCDG7LL2ZSHZSNFBCDLW65HNK5HOXT3ORXQEH5WPRADICA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.759993346 (74h26m7.799566466s) [noise=true , address=45.91.116.4:4937 , piece_id=E56MCYNPR6NMSX2ZQ7ERY6OCWRHDHSZAM4WLMRM6WKXUOZ5XEMPA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.779235778 (74h26m7.818808898s) [noise=true , address=45.91.116.6:4973 , piece_id=E3V6FUPMCI67SNXVRF6OQILEY7S737NF5FDPQDECBA2M6YDRXP2Q, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.865162765 (74h26m7.904735885s) [noise=true , address=45.91.116.10:4083 , piece_id=MGM4SPIZGII4DPBRNJ335LX372ORMFUGIF2NI55BNQ5FNNWSWINA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.942184412 (74h26m7.981757532s) [noise=true , address=77.90.31.15:4699 , piece_id=UQ4OYU7ZRNG2P4HPOFJH3VOQX4ZPN5XUUIT34CPQAASKTWGEGBOA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.950171005 (74h26m7.989744125s) [noise=true , address=45.91.116.4:4939 , piece_id=KIEAXZP5NLYZHY6WYEKH7JEKMBL2LYZ2T6XJPCL6NAU77AIXE6EA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.953503222 (74h26m7.993076342s) [noise=true , address=77.90.31.17:4770 , piece_id=IF4XNFYZXLUUZZHRGNUZOKM7AJVWJ6RDOIEGRIW4YGXKTHPDQROQ, seg_pos={0 0}, upload=65495]: Done uploading piece. err: <nil>
Sep 9 15:54:31.953515064 (74h26m7.993088184s) [seg_pos={0 0}, upload=65495]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:54:31.953618452 (74h26m7.993191572s) [noise=true , address=145.79.227.7:4294 , piece_id=OR2GKH5WITNRCE7IHE6KJZHDLU6FWHKQR5GXSKTCMPWGFIN5E6UQ, seg_pos={0 0}, upload=65495]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF
Sep 9 15:54:31.953676102 (74h26m7.993249222s) [noise=true , address=45.91.116.5:4298 , piece_id=NQ3IIUXCWW2JBODUZAVSTTCU3AI4LBXEGSV7UWSGZK4GSACTNQSA, seg_pos={0 0}, upload=65495]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF
Sep 9 15:54:31.953818706 (74h26m7.993391826s) [noise=true , address=77.90.31.18:4806 , piece_id=O6MX7VK646TNIN6D6MFVXVUSZ7W3UK63JAK53DKV6M7H5FBWPQ7A, seg_pos={0 0}, upload=65495]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:54696->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:54696->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54696->77.90.31.18:4806: use of closed network connection
Sep 9 15:54:31.953856037 (74h26m7.993429157s) [seg_pos={0 0}, upload=65495]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:54:31.953929127 (74h26m7.993502247s) [upload=65495]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:54:31.953932834 (74h26m7.993505954s) [upload=65495]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:54:32.421925692 (74h26m8.461498822s) [upload=65495]: Done uploading segments...
Sep 9 15:54:32.421950780 (74h26m8.46152391s) [upload=65495]: Upload finished. err: <nil>
Duration: 30.001430629s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [......X.X....X...................X..........................] duration:
7.398461436s
FAIL (2025-09-09 15:54:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.398499518s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.898968281s
FAIL (2025-09-09 15:54:43 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.898992918s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.......X.X....X...................X.........................] duration:
7.718521747s
FAIL (2025-09-09 15:54:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.718566182s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.540016999s
FAIL (2025-09-09 15:54:13 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.540042558s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
52 out of 60 is failed (threshold=3) [X.XX..XX.X.XX..X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.002214473s
FAIL (2025-09-09 15:54:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
Sep 9 15:53:32.650688518 (74h25m8.690261638s) [upload=65482]: Starting upload
Sep 9 15:53:32.650736630 (74h25m8.69030975s) [upload=65482]: Done starting upload
Sep 9 15:53:32.650901255 (74h25m8.690474375s) [upload=65482]: Uploading segments...
Sep 9 15:53:32.650952282 (74h25m8.690525402s) [upload=65482]: Got next segment. Inline: false
Sep 9 15:53:32.650986378 (74h25m8.690559508s) [upload=65482]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 9 15:53:32.650990746 (74h25m8.690563866s) [upload=65482]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 9 15:53:32.912214627 (74h25m8.951787747s) [seg_pos={0 0}, upload=65482]: Begin upload segment...
Sep 9 15:53:32.912262379 (74h25m8.951835499s) [seg_pos={0 0}, upload=65482]: Done begin upload segment.
Sep 9 15:53:32.912284692 (74h25m8.951857812s) [noise=true , address=45.91.116.5:4297 , piece_id=3UECPTTXHV6ONT4VFIDDJLYZJ4XMD7IAC5AERGZDOMCGKE4X2T7A, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.912343083 (74h25m8.951916214s) [noise=true , address=45.91.116.7:4009 , piece_id=Y2VBQTO75T4Q4PEEA2XIHFVTGGVNLR7NE6RBXUEM52V2XK72TWJA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.912653597 (74h25m8.952226717s) [noise=true , address=77.90.31.18:4805 , piece_id=47AJ47BYDKHBFKZ2KWDTC32XFHKA2ICVC7PLJ6JRUD7G7KXUGI4Q, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.912923233 (74h25m8.952496353s) [noise=true , address=45.91.116.8:4048 , piece_id=CEULFGM6YFLRQS3MDRS62H4Y37GXLXRSTIEYMYLTH5KZQO7FXQWQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.912990783 (74h25m8.952563903s) [noise=true , address=45.91.116.4:4940 , piece_id=ESI26LZXVZBU43W5BMVDD6BZEFGUB3Q3SK65O2UK2GY2ORRXI4MQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.913211986 (74h25m8.952785106s) [noise=true , address=45.91.116.14:4191 , piece_id=22OJHZBUK3KS75747CK4B3FUFBU4ECRODQLLCUTH5CTMUPEGPSRQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.913455963 (74h25m8.953029083s) [noise=true , address=77.90.31.13:4627 , piece_id=VFSXVVIFOPWOUOL5OLO4SEW6PYVGEXYCBVGFUFUX52YP2CU3GPHA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.913496200 (74h25m8.95306932s) [noise=true , address=145.79.227.14:4839 , piece_id=5JQ75H32EV2QAZ4W3BZGB34V2QMDKS2SVETK5UDINZSCGUSWFBSQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.913726461 (74h25m8.953299581s) [noise=true , address=145.79.227.17:4969 , piece_id=UBDRTMHTXHJZXPJXIXKJFMHWKE6RM2BCQWZQC7QLB7LNNOWKJYBQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.913966219 (74h25m8.953539339s) [noise=true , address=77.90.31.13:4628 , piece_id=DRB6D6GLLD5YKWLQCA6OZC5Y66WRV52P7ITJMW3XRUZUA6RFPSGA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.914209996 (74h25m8.953783116s) [noise=true , address=45.91.116.9:4263 , piece_id=KIHWVAHPGTNH4AZGUFIWYKISB35AM57MLJE44T2NKKG5C2KBLUTA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.914460485 (74h25m8.954033605s) [noise=true , address=45.91.116.11:4117 , piece_id=3SBTPVO3ZMKP77CFF5UFBDC3HSXJPPP2HYFLJ2DL4VVGNXMA7FDA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.914689133 (74h25m8.954262253s) [noise=true , address=145.79.227.15:4875 , piece_id=LXTTM65COP5Y5EYTNRKU54HXDLM3K2FH6LTLZ3TYTIWG4MSAJ5UQ, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.914826546 (74h25m8.954399666s) [noise=true , address=45.91.116.12:4225 , piece_id=QIODAH4WU57GB6NJD2E3K6VORQFJNAGM45QGQVJZOCQYEFWBYDMA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.915188658 (74h25m8.954761778s) [noise=true , address=77.90.31.19:4843 , piece_id=BPHTPNFJ25CLGLNYC3XYKBQ6NTCRBIL7QX256UWG2ABZDRQXFAFA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:32.915596729 (74h25m8.955169849s) [noise=true , address=77.90.31.21:4877 , piece_id=4SEHZ5RM4RLE432I7PLH7AMP7FKKC6NXL2RSCRD27JJMIJBQFHIA, seg_pos={0 0}, upload=65482]: Uploading piece...
Sep 9 15:53:34.796056646 (74h25m10.835629767s) [upload=65482]: Next returned nil segment
Sep 9 15:53:34.796074511 (74h25m10.835647631s) [upload=65482]: Waiting for error group managing segments...
Sep 9 15:53:35.097503207 (74h25m11.137076327s) [noise=true , address=45.91.116.8:4048 , piece_id=CEULFGM6YFLRQS3MDRS62H4Y37GXLXRSTIEYMYLTH5KZQO7FXQWQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.110164024 (74h25m11.149737144s) [noise=true , address=45.91.116.11:4117 , piece_id=3SBTPVO3ZMKP77CFF5UFBDC3HSXJPPP2HYFLJ2DL4VVGNXMA7FDA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.117553344 (74h25m11.157126464s) [noise=true , address=77.90.31.19:4843 , piece_id=BPHTPNFJ25CLGLNYC3XYKBQ6NTCRBIL7QX256UWG2ABZDRQXFAFA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.137472861 (74h25m11.177045981s) [noise=true , address=45.91.116.9:4263 , piece_id=KIHWVAHPGTNH4AZGUFIWYKISB35AM57MLJE44T2NKKG5C2KBLUTA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.152743934 (74h25m11.192317054s) [noise=true , address=45.91.116.12:4225 , piece_id=QIODAH4WU57GB6NJD2E3K6VORQFJNAGM45QGQVJZOCQYEFWBYDMA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.192050379 (74h25m11.231623499s) [noise=true , address=45.91.116.4:4940 , piece_id=ESI26LZXVZBU43W5BMVDD6BZEFGUB3Q3SK65O2UK2GY2ORRXI4MQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.225244059 (74h25m11.264817179s) [noise=true , address=77.90.31.21:4877 , piece_id=4SEHZ5RM4RLE432I7PLH7AMP7FKKC6NXL2RSCRD27JJMIJBQFHIA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.226446570 (74h25m11.26601969s) [noise=true , address=77.90.31.18:4805 , piece_id=47AJ47BYDKHBFKZ2KWDTC32XFHKA2ICVC7PLJ6JRUD7G7KXUGI4Q, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.255479525 (74h25m11.295052646s) [noise=true , address=77.90.31.13:4628 , piece_id=DRB6D6GLLD5YKWLQCA6OZC5Y66WRV52P7ITJMW3XRUZUA6RFPSGA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.304487085 (74h25m11.344060205s) [noise=true , address=77.90.31.13:4627 , piece_id=VFSXVVIFOPWOUOL5OLO4SEW6PYVGEXYCBVGFUFUX52YP2CU3GPHA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.453615343 (74h25m11.493188463s) [noise=true , address=45.91.116.14:4191 , piece_id=22OJHZBUK3KS75747CK4B3FUFBU4ECRODQLLCUTH5CTMUPEGPSRQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.724741643 (74h25m11.764314763s) [noise=true , address=145.79.227.15:4875 , piece_id=LXTTM65COP5Y5EYTNRKU54HXDLM3K2FH6LTLZ3TYTIWG4MSAJ5UQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.825628237 (74h25m11.865201357s) [noise=true , address=145.79.227.14:4839 , piece_id=5JQ75H32EV2QAZ4W3BZGB34V2QMDKS2SVETK5UDINZSCGUSWFBSQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: <nil>
Sep 9 15:53:35.825657273 (74h25m11.865230393s) [seg_pos={0 0}, upload=65482]: Segment reached optimal threshold of 13 pieces.
Sep 9 15:53:35.825781250 (74h25m11.86535437s) [noise=true , address=45.91.116.5:4297 , piece_id=3UECPTTXHV6ONT4VFIDDJLYZJ4XMD7IAC5AERGZDOMCGKE4X2T7A, seg_pos={0 0}, upload=65482]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Sep 9 15:53:35.825871823 (74h25m11.865444943s) [noise=true , address=45.91.116.7:4009 , piece_id=Y2VBQTO75T4Q4PEEA2XIHFVTGGVNLR7NE6RBXUEM52V2XK72TWJA, seg_pos={0 0}, upload=65482]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF
Sep 9 15:53:35.825918753 (74h25m11.865491863s) [noise=true , address=145.79.227.17:4969 , piece_id=UBDRTMHTXHJZXPJXIXKJFMHWKE6RM2BCQWZQC7QLB7LNNOWKJYBQ, seg_pos={0 0}, upload=65482]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF
Sep 9 15:53:35.825964120 (74h25m11.86553724s) [seg_pos={0 0}, upload=65482]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Sep 9 15:53:35.826036889 (74h25m11.865610019s) [upload=65482]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Sep 9 15:53:35.826040927 (74h25m11.865614037s) [upload=65482]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Sep 9 15:53:36.154639313 (74h25m12.194212433s) [upload=65482]: Done uploading segments...
Sep 9 15:53:36.154657438 (74h25m12.194230548s) [upload=65482]: Upload finished. err: <nil>
Duration: 30.002237888s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
680.566766ms target:
https://us1.storj.io/login
FAIL (2025-09-09 15:54:00 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 680.595611ms
core
satellite
us1
|