Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 420.094925ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:24:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 420.113529ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.63894219s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:24:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.638958039s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 386.729624ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:24:01 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 386.804864ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.672588345s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:23:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.672602191s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.584967531s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:23:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.584986536s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.633026227s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:23:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.633050072s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 379.440378ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:23:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 379.464603ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.083986029s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:22:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.084009402s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 378.626439ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:22:23 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 378.6482ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.161489254s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:22:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.161529037s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.897712861s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:22:02 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.897730755s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.166087125s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:21:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.16610599s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 388.699468ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:21:09 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 388.721039ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.80755092s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:20:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.807573082s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.239035323s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:20:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.239059669s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 389.268808ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:20:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 389.286753ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.672051056s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:19:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.67207011s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.125241761s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:19:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.125261478s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.953782891s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:19:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.953806315s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 374.956794ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:19:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 374.976312ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.129532078s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:19:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.129555541s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 396.103498ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:19:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 396.123465ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.938396867s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:18:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.938416423s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 375.29785ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:18:31 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 375.307117ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.125617063s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:18:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.125639896s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 386.898362ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:18:07 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 386.920983ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.877223089s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:17:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.877240983s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 370.637034ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:17:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 370.654146ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.146299577s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:17:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.146321427s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.676764693s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:17:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.676788166s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 360.497832ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:17:01 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 360.517088ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.684065369s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:16:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.684088772s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.909144441s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:16:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.909161693s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.17241681s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:16:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.172433471s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.160844487s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:15:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.160864875s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 372.444069ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:15:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 372.46087ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.14058993s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:15:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.140612322s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.820241201s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:15:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.820259495s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 432.428898ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:15:09 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 432.441501ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 387.613224ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:14:28 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 387.632861ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.13291722s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:14:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.132929453s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.835842404s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:14:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.835860439s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.131201315s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:13:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.131219038s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 391.526478ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:13:45 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 391.593673ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 3.070204681s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:13:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 3.070227304s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 422.772868ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:12:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.796491ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.147112919s trace: https://jaeger.dev.storj.io/trace/5683d14cc62eee41 FAIL (2025-12-16 17:12:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . Dec 16 16:53:09.845598423 (265h23m57.543166758s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846066447 (265h23m57.543634783s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.846618208 (265h23m57.544186544s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847117681 (265h23m57.544686018s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.847646829 (265h23m57.545215176s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849322711 (265h23m57.546891047s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.849923874 (265h23m57.54749221s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.850581442 (265h23m57.548149788s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851295546 (265h23m57.548863882s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:09.851846756 (265h23m57.549415132s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Uploading piece... Dec 16 16:53:10.300410080 (265h23m57.997978386s) [upload=233340]: Next returned nil segment Dec 16 16:53:10.300419738 (265h23m57.997988054s) [upload=233340]: Waiting for error group managing segments... Dec 16 16:53:10.487707706 (265h23m58.185276032s) [noise=true , address=185.167.204.206:4066 , piece_id=YAMEYYASLEP75NXGHJ65OK7HOS67BVR5ZG2WTMKNUBXCOUOBPPOQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488519382 (265h23m58.186087698s) [noise=true , address=185.167.204.206:4072 , piece_id=6BIZRUDCOCOYAF33R5OBNC7JFO24KVYSWPUBWAGQ5KDFMTBE25TA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.488981015 (265h23m58.18654932s) [noise=true , address=185.167.204.205:4020 , piece_id=FAU7CY7IYUPFV47CRJ55FLCBTLL4QUIFJ5NGKBNYR53PUCRIKDNQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.489259936 (265h23m58.186828272s) [noise=true , address=185.167.204.205:4027 , piece_id=7DJF2FS4NNSAM7O3JLCHSXDYXW2B2TK5HBRCVHHCEPXRQ3JFFOJA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.493026132 (265h23m58.190594468s) [noise=true , address=185.167.204.205:4033 , piece_id=F2X3ZTK52MNL5Q6W5FEDNRLBFBWJSMQ5XUX7HVNMRTYFRUP4NF5A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.497763182 (265h23m58.195331487s) [noise=true , address=185.167.204.206:4063 , piece_id=CXJZQGWPLX2GR6FT4PVMJLFRIFQQCLS4FOCPAJSS4ZV34IX4DN2A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.500875446 (265h23m58.198443762s) [noise=true , address=185.167.204.205:4015 , piece_id=L7P6B35RE5NTIOA3JTG535RSBF42IHHC3SJSJY7JIQGP4KJUHIYQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.503696186 (265h23m58.201264491s) [noise=true , address=89.149.201.147:4021 , piece_id=Z3CQ3EQAXFLENOPKJP3DKAYGXNO2376ST6Z55HGJSMZYEMBU73NQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.506666203 (265h23m58.20423454s) [noise=true , address=185.167.204.205:4016 , piece_id=I7ROD6VOVVTE5OOR462FPNK6FB5E6HCFVOAOATCYEGM7GEHI36MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.514568155 (265h23m58.212136492s) [noise=true , address=185.167.204.205:4030 , piece_id=YKGL25SO3PZ2LBJMNJEPB4FO7PLWTD62GM4IIUKBLDYZTMBLQDCA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.516326801 (265h23m58.213895117s) [noise=true , address=185.167.204.206:4064 , piece_id=2PTLHY2JHHYERJM7FFAZVP6EBDX5D7RW7RNQPABQLSFT55HJA2QQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.524488249 (265h23m58.222056635s) [noise=true , address=185.167.204.206:4062 , piece_id=4YYT67CBPVP5QQHT7O2IK6GVLJAFNFALH76IMWILSRTA7NVNU2YQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528223667 (265h23m58.225792004s) [noise=true , address=185.167.204.206:4054 , piece_id=D4AZ6L635EREAOFXQB5WOZODYHI3JDX622TDCIQFYAQVQNLY5PQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.528675321 (265h23m58.226243637s) [noise=true , address=89.149.201.147:4010 , piece_id=QPL4PZCA57NVK32MVFHIPRUZ7NAMFJWGZPQK7PP6QMTRP4QCNGUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.532773757 (265h23m58.230342073s) [noise=true , address=185.167.204.205:4002 , piece_id=PUYVE7TVOLH4ZH5SVTDEADQLFEYCB62PH3LPNBYJDJYNN7246VPA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.539818999 (265h23m58.237387315s) [noise=true , address=81.171.14.70:4048 , piece_id=R2KPQ7VPMPZHTH6PXZCDYN6HZVWJFA7RC55LMKZTQNCUQNQV2VAA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.550626359 (265h23m58.248194665s) [noise=true , address=81.171.14.70:4041 , piece_id=7N3BCTKBLRNEHWFDA4JG5EL2724JODUSMITRCNT5GNHHQ3XMXOGA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.562449477 (265h23m58.260017793s) [noise=true , address=81.171.14.70:4036 , piece_id=2DRO2WQNKIQOLQONGLT6UPIWA3MQPWOESLFKTWLP2ALHZPGBMKDA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.581197291 (265h23m58.278765627s) [noise=true , address=89.149.201.147:4012 , piece_id=C4N4DMRVA7VHN3MVUXQ7YCGC67Q7WAW6AMM7E35XJUXQUB3F26MQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.585618260 (265h23m58.283186567s) [noise=true , address=89.149.201.147:4009 , piece_id=SJOCJGUYVOXZWMHZQEGNMZXJDCZUMB2WP5FSYJ3EAUNQGODCOFQA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.592780691 (265h23m58.290349027s) [noise=true , address=185.167.204.206:4071 , piece_id=D3SV3RVHYGAU77TLBJNHBETNKILWBLOHOKCX5POSX36USLR7YT6A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.618165813 (265h23m58.315734129s) [noise=true , address=89.149.201.147:4008 , piece_id=JKTXCHUFHQQA5HLZU4MYASODZVSGXBTXKAFIHMEB4DZVZDSWGVKA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:10.964120185 (265h23m58.661688521s) [noise=true , address=81.171.14.70:4037 , piece_id=I574CDK3DF4VD6DNSU2EZVEZSXV7TXQLTDQ6PXOGNZ5NBS3WLG7A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.026637691 (265h23m58.724206017s) [noise=true , address=89.149.201.147:4017 , piece_id=WV2W3UABBSPQTLLCYXM2WAKP2I5BUFE4M4XD3WECWR7EBUEOYLHQ, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.033231919 (265h23m58.730800256s) [noise=true , address=89.149.201.147:4006 , piece_id=TYU2VBU2E4CBT4WA7LWBTSK6WMJNOY753UFSCLPRFNQUXUMGBXUA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040392117 (265h23m58.737960453s) [noise=true , address=81.171.14.70:4043 , piece_id=WGFJVAFZ7U52TS35UNWIMB2NY66KOXYV5DEI5PCURD2D4VFWJXIA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: <nil> Dec 16 16:53:11.040410391 (265h23m58.737978737s) [seg_pos={0 0}, upload=233340]: Segment reached optimal threshold of 26 pieces. Dec 16 16:53:11.040665407 (265h23m58.738233743s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040681107 (265h23m58.738249453s) [noise=true , address=89.149.201.147:4013 , piece_id=K3QBGLIOZ3LUYJV24CNESXAEJVSTM4BREEY2NTRNVBGZYTESTK4A, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.040938978 (265h23m58.738507314s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2): protocol: expected piece hash; context canceled; EOF Dec 16 16:53:11.040951392 (265h23m58.738519727s) [noise=true , address=89.149.201.147:4019 , piece_id=QNHI7H4QBIYTA64JBQJKFPJEHK54WGHWEXOPTNME7ZFZ2FJNT7GA, seg_pos={0 0}, upload=233340]: Long tail cancellation due to optimal threshold Dec 16 16:53:11.041003759 (265h23m58.738572096s) [seg_pos={0 0}, upload=233340]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 16 16:53:11.041087015 (265h23m58.738655351s) [upload=233340]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 16 16:53:11.041093397 (265h23m58.738661733s) [upload=233340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 16 16:54:06.282124914 (265h24m53.97969323s) [upload=233340]: Done uploading segments... Dec 16 16:54:06.282202999 (265h24m53.979771305s) [upload=233340]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.147135731s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 3.018761187s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:12:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 3.018785392s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.983012656s trace: https://jaeger.dev.storj.io/trace/7608b45d5a11c8a3 FAIL (2025-12-16 17:11:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.983065304s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 410.485574ms trace: https://jaeger.dev.storj.io/trace/1054383a00e42cf4 FAIL (2025-12-16 17:11:49 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 410.518847ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)