Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 866405 reports (Page 11667 of 17329)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.......X......X................X.XX.....X...................] duration: 5.992752451s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:18:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.992781757s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.122434581s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:18:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.122464818s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.X.X...X....X..........X.........X..........................] duration: 3.610359283s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:18:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.610384923s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [....X...............................X.......................] duration: 488.424846ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:18:22 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 488.459301ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.906494071s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:18:19 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.906522345s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.....X...............................X......................] duration: 2.377303954s remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:18:00 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 2.377331627s core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.757942877s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:17:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.757972744s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [..X.X...X....X..........X.........X.........................] duration: 3.524493635s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:17:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.524525416s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [........X......X................X.XX.....X..................] duration: 5.721545535s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:17:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.721580191s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.880413919s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:17:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.880439437s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.........X......X................X.XX.....X.................] duration: 5.894102114s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:17:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.8941319s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 10.043002296s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:17:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 10.043027944s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [...X.X...X....X..........X.........X........................] duration: 3.606944975s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:17:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.606972407s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [......X...............................X.....................] duration: 485.542977ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:17:14 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 485.588235ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [....X.X...X....X..........X.........X.......................] duration: 3.526752458s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:16:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.526770441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.851630626s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:16:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.851657507s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.....X.X...X....X..........X.........X......................] duration: 3.511996918s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:16:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.512026014s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.......X...............................X....................] duration: 472.156418ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:16:27 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 472.18297ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..........X......X................X.XX.....X................] duration: 5.672078334s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:16:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.672110076s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.274517246s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:16:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.274547804s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [......X.X...X....X..........X.........X.....................] duration: 3.451409435s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:16:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.451437247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.83655576s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:16:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.836580789s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.125961122s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:15:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.12599178s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...........X......X................X.XX.....X...............] duration: 5.909916338s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:15:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.909941867s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.739200542s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:15:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.739229928s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.......X.X...X....X..........X.........X....................] duration: 4.465818766s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:15:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.465845677s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [............X......X................X.XX.....X..............] duration: 5.847438994s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:15:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.847469603s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [........X...............................X...................] duration: 474.39958ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:15:16 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 474.430921ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.91719595s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:15:05 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.917230817s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.........X...............................X..................] duration: 491.376015ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:15:00 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 491.410451ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.17298854s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:14:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.173020139s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..........X...............................X.................] duration: 484.010359ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:14:35 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 484.046348ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [........X.X...X....X..........X.........X...................] duration: 3.54397491s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:14:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.544005717s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.898961859s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:14:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.898990474s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.............X......X................X.XX.....X.............] duration: 5.580783039s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:14:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.580814818s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.375396068s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:14:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.375558859s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.........X.X...X....X..........X.........X..................] duration: 3.602976468s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:14:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.603002096s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.707236578s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:13:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 5.707254472s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..............X......X................X.XX.....X............] duration: 5.696139436s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:13:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.69616751s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...........X...............................X................] duration: 488.00129ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:13:48 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 488.060903ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.002215688s trace: https://jaeger.dev.storj.io/trace/221485e4375dc5e9 FAIL (2025-12-09 00:13:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece... Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments... Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil> Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces. Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments... Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil> Duration: 1m0.00223832s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.476973894s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:13:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.476998101s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 34.899816655s trace: https://jaeger.dev.storj.io/trace/7bce72a308ed627a FAIL (2025-12-09 00:12:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 34.89984526s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [..........X.X...X....X..........X.........X.................] duration: 3.473370573s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:12:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.473396773s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [............X...............................X...............] duration: 483.054616ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:12:44 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 483.084112ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...............X......X................X.XX.....X...........] duration: 5.684499553s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:12:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.684669517s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [X..........X.X...X....X..........X.........X................] duration: 3.503812563s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:11:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503828313s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [................X......X................X.XX.....X..........] duration: 5.629531784s trace: https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66 FAIL (2025-12-09 00:11:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.629551833s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.............X...............................X..............] duration: 484.976305ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735 FAIL (2025-12-09 00:11:30 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 484.995061ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.X..........X.X...X....X..........X.........X...............] duration: 3.435484739s trace: https://jaeger.dev.storj.io/trace/424de39ee362be3f FAIL (2025-12-09 00:11:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.435527712s core uplink us-select-1 us1-placement-20

1
...
11665
11666
11667
11668
11669
...
17329