Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..................X.........X......X........................] duration:
2.582064236s
FAIL (2025-11-28 23:14:25 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.582088662s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.169958438s
FAIL (2025-11-28 23:13:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.169979678s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
500.085665ms
FAIL (2025-11-28 23:13:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 500.101905ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...................X.........X......X.......................] duration:
2.848468277s
FAIL (2025-11-28 23:13:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.848487433s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.585926236s
FAIL (2025-11-28 23:13:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.585947175s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................X.........X......X......................] duration:
2.712683471s
FAIL (2025-11-28 23:12:47 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.712702346s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
485.444397ms
FAIL (2025-11-28 23:12:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 485.463062ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
472.768189ms
FAIL (2025-11-28 23:12:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 472.985657ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....................X.........X......X.....................] duration:
2.762302781s
FAIL (2025-11-28 23:12:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.762318912s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.611906178s
FAIL (2025-11-28 23:12:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.611925283s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
403.653937ms
FAIL (2025-11-28 23:11:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 403.669556ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
505.781059ms
FAIL (2025-11-28 23:11:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 506.299581ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......................X.........X......X....................] duration:
2.849298872s
FAIL (2025-11-28 23:11:36 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.849325512s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
469.165448ms
FAIL (2025-11-28 23:10:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 469.210613ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
384.159246ms
FAIL (2025-11-28 23:10:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 384.178011ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.150263715s
FAIL (2025-11-28 23:10:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.150279655s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......................X.........X......X...................] duration:
2.753118119s
FAIL (2025-11-28 23:10:49 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.753138117s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
388.435845ms
FAIL (2025-11-28 23:10:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 388.481692ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [........................X.........X......X..................] duration:
2.863088249s
FAIL (2025-11-28 23:10:14 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.863114439s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
508.942413ms
FAIL (2025-11-28 23:09:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 508.96123ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.426635998s
FAIL (2025-11-28 23:09:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.426657569s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
396.361223ms
FAIL (2025-11-28 23:09:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 396.381431ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................X.........X......X.................] duration:
2.794478309s
FAIL (2025-11-28 23:09:04 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.794501532s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.808646411s
FAIL (2025-11-28 23:08:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.808667851s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
458.203403ms
FAIL (2025-11-28 23:08:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 458.229774ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........................X.........X......X................] duration:
2.885612067s
FAIL (2025-11-28 23:08:18 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.885628778s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.727702095s
FAIL (2025-11-28 23:08:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.72772077s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
331.857931ms
FAIL (2025-11-28 23:08:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 331.876987ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.869098919s
FAIL (2025-11-28 23:07:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.869127874s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
398.707563ms
FAIL (2025-11-28 23:07:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 398.728262ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
493.156896ms
FAIL (2025-11-28 23:07:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 493.220255ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........................X.........X......X...............] duration:
2.768696138s
FAIL (2025-11-28 23:07:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.76871793s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [............................X.........X......X..............] duration:
2.892699725s
FAIL (2025-11-28 23:06:47 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.892719794s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
416.510341ms
FAIL (2025-11-28 23:06:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 416.530099ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.720707761s
FAIL (2025-11-28 23:06:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.720729651s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
490.573156ms
FAIL (2025-11-28 23:06:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 490.594856ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............................X.........X......X.............] duration:
3.204145921s
FAIL (2025-11-28 23:06:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 3.20416706s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.172190695s
FAIL (2025-11-28 23:06:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.172219227s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
345.6329ms
FAIL (2025-11-28 23:06:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 345.65354ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
491.775829ms
FAIL (2025-11-28 23:05:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 491.798711ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.730801167s
FAIL (2025-11-28 23:05:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.730826545s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............................X.........X......X............] duration:
3.153252918s
FAIL (2025-11-28 23:05:17 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 3.153270791s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
350.518426ms
FAIL (2025-11-28 23:05:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 350.534878ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
491.024483ms
FAIL (2025-11-28 23:05:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 491.043037ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.427331269s
FAIL (2025-11-28 23:05:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.427353591s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
3.089054108s
FAIL (2025-11-28 23:04:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.089076641s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
336.014794ms
FAIL (2025-11-28 23:04:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 336.031755ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.747468807s
FAIL (2025-11-28 23:04:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.333604686 (779h45m7.034549581s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334038239 (779h45m7.034983134s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334478604 (779h45m7.035423489s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.334902039 (779h45m7.035846923s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335378582 (779h45m7.036323477s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.335794502 (779h45m7.036739387s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336219879 (779h45m7.037164774s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.336703296 (779h45m7.037648201s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337090612 (779h45m7.038035497s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.337503015 (779h45m7.03844791s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Uploading piece...
Nov 28 22:58:40.782003596 (779h45m7.482948491s) [upload=686083]: Next returned nil segment
Nov 28 22:58:40.782024205 (779h45m7.4829691s) [upload=686083]: Waiting for error group managing segments...
Nov 28 22:58:40.968853459 (779h45m7.669798354s) [noise=true , address=185.167.204.205:4009 , piece_id=BJD24GMUQVWLTTAXPMHBRZXT665RDACHQRWNVKB4HHKOQRS3YU3A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969034238 (779h45m7.669979122s) [noise=true , address=185.167.204.205:4002 , piece_id=GBLKBRVRWUJF3B2G2T3NWHSDW3ZBTKIPA5AGDE6MX3FDPFZESITQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969225757 (779h45m7.670170652s) [noise=true , address=185.167.204.205:4029 , piece_id=BTJKS7EZI2F7E7KSWJ25IAS5VOL6VI6HWMBLAGTULWEAOFSQKXCA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.969825742 (779h45m7.670770627s) [noise=true , address=185.167.204.206:4069 , piece_id=QHOI5OUTEQWIFDVYXHTV4VZBN4EWMT2HT5FV642LP2ZJVGTF242A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.970117920 (779h45m7.671062805s) [noise=true , address=81.171.14.70:4041 , piece_id=YLTIQRVI7WGEPUD52NN4UTWPKFJJK6VKBEC3VQY2CVTZKABHYIFA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.971205448 (779h45m7.672150334s) [noise=true , address=185.167.204.206:4058 , piece_id=ANQXN4JHSWZLY6CTVM3CMUR6LHC27UE4OJVXSTNW7O2WPEZADXQQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.973311928 (779h45m7.674256813s) [noise=true , address=89.149.201.147:4006 , piece_id=BDLPZN4YICYLXU5ZGKCL6SNTSHV73SIFEHJ5EYIEOVWXIQRWGYEA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975029258 (779h45m7.675974153s) [noise=true , address=185.167.204.205:4026 , piece_id=XG45SBLVIZS4IQR46V3M23PFCJ6YK5KAWT527KNZEG57MHLWLDLA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.975667334 (779h45m7.676612229s) [noise=true , address=185.167.204.206:4048 , piece_id=GJSEMMLHBOGB3UIPU576ODIFDUCJIZNTXQCMOQ3CI33DJ6IRG5AQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.978678009 (779h45m7.679622895s) [noise=true , address=185.167.204.206:4050 , piece_id=NQKQRUGTFYQYC7BNNRNYX4CLIFBN3NCKPAQY3FZBLZY6I7ZUESNA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.984808924 (779h45m7.685753819s) [noise=true , address=185.167.204.206:4068 , piece_id=KGFSHPP4HBZK6E5SVIC462TDHL6Q7TNEPJOX7VP25JEBUQMYMR6A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.985104939 (779h45m7.686049834s) [noise=true , address=185.167.204.205:4022 , piece_id=5KF2EPH2ES2CKJFTNKJ2R4NYI7U3OD4DDMYAZSMBRKHFUBENTZKQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:40.987514928 (779h45m7.688459823s) [noise=true , address=185.167.204.205:4005 , piece_id=HIVUROV3ZIVICBQ7ZXGJFVLDB57L2HEIL3XKZ4OVQYLSEJM6C4QQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.000829422 (779h45m7.701774307s) [noise=true , address=185.167.204.206:4067 , piece_id=GKKNX7LRROSJEUQYDPMDYLBL3RCJV6QXTVWAZRJIHN5FFFVOVAHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.003293562 (779h45m7.704238467s) [noise=true , address=185.167.204.206:4071 , piece_id=VZTSQUJWTVXGLXXK3FCFDQ6DKPKN6YHEZJ5SGUUE6YC4HYVXA2JA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.008807219 (779h45m7.709752104s) [noise=true , address=89.149.201.147:4013 , piece_id=QXZCHLFFBS75E5AID3KLVGZIJC6EE5TL3OOXXYAS2GHAYR47GWRA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.011329949 (779h45m7.712274845s) [noise=true , address=185.167.204.205:4012 , piece_id=ZY4KSBPPAPVTN5K6NXFFYE3XJMUOOGX2RCKRI2CS7ZLPZQRY75VA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.012215108 (779h45m7.713160003s) [noise=true , address=185.167.204.206:4044 , piece_id=E4PPZWCOTXEO2PMUA7SEAJEDH37KSJLYMGCD65EZV74VCZELQJIA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014091136 (779h45m7.715036031s) [noise=true , address=185.167.204.206:4041 , piece_id=6YZGKJ4PEBYEBSZC3AYOFHBQD75Y4TTX6DOANQ77VMWIW6HOAJEQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.014974904 (779h45m7.715919799s) [noise=true , address=185.167.204.205:4028 , piece_id=YPC2VUYP6OU2K2CBO4RBVZCN62VTUTGLBARKZ7EVOO3O33KISFZA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.022602245 (779h45m7.72354714s) [noise=true , address=185.167.204.205:4017 , piece_id=CNGWFNDW6XFWX6U7W4QQVWW224WXZXNRLHUE2LYH4DHXH63K7DCQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.024682966 (779h45m7.725627861s) [noise=true , address=89.149.201.147:4018 , piece_id=IZNUTX4GD3VLILISM3AIDIJEDW33WKBK6KJM32CKZGZLDE4IUPQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.029699522 (779h45m7.730644407s) [noise=true , address=81.171.14.70:4042 , piece_id=BNPUBM2IKBNBBSHKHD6QQYZDVD2GGCIYGVSEOO3WZAE3P36Y4GFQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.054942019 (779h45m7.755886904s) [noise=true , address=185.167.204.206:4053 , piece_id=RNFCQME54W3K4IFVB4LGCZSSHDC7WABDPBJMM7A3KYLDSEQZBUQA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.109618094 (779h45m7.810562989s) [noise=true , address=89.149.201.147:4016 , piece_id=BEUWAIFX7IVS6E2IADFEA3ROJBY5TP372VRS37CK2W7QWHV4GM3Q, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481890353 (779h45m8.182835268s) [noise=true , address=185.167.204.206:4047 , piece_id=JZZRQH4OA7ATQX7SRCVOM26PQHGXQVZHPY7TZV7PRW4CMZUQKG5A, seg_pos={0 0}, upload=686083]: Done uploading piece. err: <nil>
Nov 28 22:58:41.481926521 (779h45m8.182871436s) [seg_pos={0 0}, upload=686083]: Segment reached optimal threshold of 26 pieces.
Nov 28 22:58:41.482063928 (779h45m8.183008834s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482073757 (779h45m8.183018672s) [noise=true , address=89.149.201.147:4002 , piece_id=ZN4MDDCO42UEDD6VSZ5CSCIX4TFB5HQQX4TVDZBO2QKBEZTCKEHA, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482189363 (779h45m8.183134258s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: expected piece hash; context canceled; EOF
Nov 28 22:58:41.482206996 (779h45m8.183151891s) [noise=true , address=89.149.201.147:4010 , piece_id=OVXNP7E72364TYPL7HLAC6EJRC4FO6IXJLJ6UPVN5H5UWIGXTPGQ, seg_pos={0 0}, upload=686083]: Long tail cancellation due to optimal threshold
Nov 28 22:58:41.482247863 (779h45m8.183192758s) [seg_pos={0 0}, upload=686083]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 28 22:58:41.482341579 (779h45m8.183286474s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 28 22:58:41.482348242 (779h45m8.183293127s) [upload=686083]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 28 22:59:38.930514200 (779h46m5.631459086s) [upload=686083]: Done uploading segments...
Nov 28 22:59:38.930532254 (779h46m5.631477149s) [upload=686083]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.747495929s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...............................X.........X......X...........] duration:
2.823111034s
FAIL (2025-11-28 23:04:18 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.823140781s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
347.33396ms
FAIL (2025-11-28 23:04:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 347.355581ms
edge
gateway
us-select-1
|