Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 520.793488ms FAIL (2025-12-03 03:47:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 520.809608ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 474.83647ms FAIL (2025-12-03 03:46:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.860735ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 450.278876ms FAIL (2025-12-03 03:46:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 450.305536ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration: 514.2619ms FAIL (2025-12-03 03:46:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 514.279394ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 409.002688ms FAIL (2025-12-03 03:45:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 409.027756ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [......X.X...................................................] duration: 481.292263ms FAIL (2025-12-03 03:45:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.308584ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 454.914453ms FAIL (2025-12-03 03:45:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 454.93388ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration: 496.332205ms FAIL (2025-12-03 03:44:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 496.352082ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 412.908498ms FAIL (2025-12-03 03:44:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 412.927876ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 396.757591ms FAIL (2025-12-03 03:43:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 396.779522ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [........X.X.................................................] duration: 527.656398ms FAIL (2025-12-03 03:43:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 527.674762ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 396.243356ms FAIL (2025-12-03 03:42:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 396.264755ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.........X.X................................................] duration: 489.212286ms FAIL (2025-12-03 03:42:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 489.231703ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 444.235305ms FAIL (2025-12-03 03:41:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 444.258058ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration: 457.114022ms FAIL (2025-12-03 03:41:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 457.132236ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 398.092556ms FAIL (2025-12-03 03:40:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 398.113385ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration: 518.888975ms FAIL (2025-12-03 03:40:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 518.936593ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 403.446023ms FAIL (2025-12-03 03:39:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 403.469527ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [............X.X.............................................] duration: 509.162246ms FAIL (2025-12-03 03:39:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 509.181831ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 422.321553ms FAIL (2025-12-03 03:39:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 422.353573ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.............X.X............................................] duration: 496.997592ms FAIL (2025-12-03 03:39:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 497.016578ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 400.872301ms FAIL (2025-12-03 03:39:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 400.899592ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration: 479.602673ms FAIL (2025-12-03 03:38:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.620055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 339.830393ms FAIL (2025-12-03 03:37:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 339.853296ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration: 476.515226ms FAIL (2025-12-03 03:37:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.534273ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 352.114637ms FAIL (2025-12-03 03:37:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 352.138562ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [................X.X.........................................] duration: 739.091561ms FAIL (2025-12-03 03:37:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 739.10636ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 373.291895ms FAIL (2025-12-03 03:36:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 373.327533ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.................X.X........................................] duration: 493.007178ms FAIL (2025-12-03 03:36:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 493.029721ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 415.831193ms FAIL (2025-12-03 03:36:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 415.852484ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration: 654.686462ms FAIL (2025-12-03 03:35:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 654.708654ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 632.370084ms FAIL (2025-12-03 03:35:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 632.39438ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...................X.X......................................] duration: 476.480604ms FAIL (2025-12-03 03:34:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.496313ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 469.522607ms FAIL (2025-12-03 03:34:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 469.5457ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration: 473.432835ms FAIL (2025-12-03 03:34:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 473.454987ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration: 464.740193ms FAIL (2025-12-03 03:33:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.757355ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 382.000509ms FAIL (2025-12-03 03:33:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 382.016739ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 402.716282ms FAIL (2025-12-03 03:33:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 402.746109ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [......................X.X...................................] duration: 477.575699ms FAIL (2025-12-03 03:32:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 477.597931ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 449.897962ms FAIL (2025-12-03 03:32:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 449.922749ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration: 504.50518ms FAIL (2025-12-03 03:31:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 504.539966ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 396.420168ms FAIL (2025-12-03 03:30:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 396.448151ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [........................X.X.................................] duration: 511.568023ms FAIL (2025-12-03 03:30:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 511.597167ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 385.496227ms FAIL (2025-12-03 03:30:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 385.514561ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.X................................] duration: 513.494658ms FAIL (2025-12-03 03:30:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.507953ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 386.927744ms FAIL (2025-12-03 03:29:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 386.955476ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.X...............................] duration: 465.688862ms FAIL (2025-12-03 03:29:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 465.712235ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 335.728928ms FAIL (2025-12-03 03:28:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 335.748144ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 494.038ms FAIL (2025-12-03 03:28:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 494.062675ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.X..............................] duration: 452.963762ms FAIL (2025-12-03 03:28:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.98437ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)