Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 332.895712ms FAIL (2025-12-03 03:27:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 332.916742ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 429.619376ms FAIL (2025-12-03 03:27:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 429.640185ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [............................X.X.............................] duration: 472.744278ms FAIL (2025-12-03 03:27:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.762073ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.X............................] duration: 443.233484ms FAIL (2025-12-03 03:26:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 443.255836ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 440.968634ms FAIL (2025-12-03 03:26:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 440.993881ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.X...........................] duration: 476.042759ms FAIL (2025-12-03 03:26:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.060633ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.X..........................] duration: 480.335787ms FAIL (2025-12-03 03:25:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.355685ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 439.234759ms FAIL (2025-12-03 03:25:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 439.266809ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [................................X.X.........................] duration: 517.224017ms FAIL (2025-12-03 03:24:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 517.240679ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 469.562572ms FAIL (2025-12-03 03:24:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 469.583201ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.X........................] duration: 480.135052ms FAIL (2025-12-03 03:24:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.151011ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 437.299889ms FAIL (2025-12-03 03:24:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 437.31592ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 620.0673ms FAIL (2025-12-03 03:23:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 620.091755ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.X.......................] duration: 470.160642ms FAIL (2025-12-03 03:23:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.222468ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 462.553365ms FAIL (2025-12-03 03:22:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 462.576621ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.X......................] duration: 460.52016ms FAIL (2025-12-03 03:22:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 460.539557ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 530.049868ms FAIL (2025-12-03 03:21:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 530.066048ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 506.489744ms FAIL (2025-12-03 03:21:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 506.516074ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 499.271489ms FAIL (2025-12-03 03:21:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 499.299181ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 363.260912ms FAIL (2025-12-03 03:20:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 363.284977ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 481.117352ms FAIL (2025-12-03 03:20:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.133754ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 484.536347ms FAIL (2025-12-03 03:19:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.596421ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 427.674445ms FAIL (2025-12-03 03:19:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 427.701294ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.X.................] duration: 568.020543ms FAIL (2025-12-03 03:18:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 568.037545ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 466.901584ms FAIL (2025-12-03 03:18:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 466.941929ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 493.577945ms FAIL (2025-12-03 03:18:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 493.593765ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.X................] duration: 525.627759ms FAIL (2025-12-03 03:18:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.639611ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 425.620321ms FAIL (2025-12-03 03:17:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 425.64648ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.X...............] duration: 512.625189ms FAIL (2025-12-03 03:16:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 512.648251ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 393.05887ms FAIL (2025-12-03 03:16:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 393.113923ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 407.429015ms FAIL (2025-12-03 03:16:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 407.451708ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.X..............] duration: 471.355896ms FAIL (2025-12-03 03:15:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 471.372637ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 405.952896ms FAIL (2025-12-03 03:15:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 405.969889ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.X.............] duration: 495.111472ms FAIL (2025-12-03 03:14:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 495.134325ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 371.701141ms FAIL (2025-12-03 03:14:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 371.712512ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 452.591156ms FAIL (2025-12-03 03:13:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 452.609149ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.X............] duration: 474.068522ms FAIL (2025-12-03 03:13:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.093137ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 448.900471ms FAIL (2025-12-03 03:13:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 448.933503ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.X...........] duration: 487.454239ms FAIL (2025-12-03 03:13:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 487.601775ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.X..........] duration: 461.024493ms FAIL (2025-12-03 03:12:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 461.061021ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 500.149693ms FAIL (2025-12-03 03:12:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 500.164901ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.X.........] duration: 463.611309ms FAIL (2025-12-03 03:12:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.629392ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 362.843839ms FAIL (2025-12-03 03:12:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 362.868967ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.X........] duration: 495.217367ms FAIL (2025-12-03 03:11:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 495.234409ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 436.529478ms FAIL (2025-12-03 03:11:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 436.550787ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration: 493.217391ms FAIL (2025-12-03 03:11:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 493.238331ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 406.419001ms FAIL (2025-12-03 03:10:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 406.442395ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.X......] duration: 498.463818ms FAIL (2025-12-03 03:10:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 498.485119ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 422.978588ms FAIL (2025-12-03 03:09:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== ZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080302860 (879h51m26.781247775s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080400042 (879h51m26.781344947s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.080935558 (879h51m26.781880473s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.081500607 (879h51m26.782445523s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082102287 (879h51m26.783047202s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.082736237 (879h51m26.783681152s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083237186 (879h51m26.784182102s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.083737716 (879h51m26.784682621s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084351898 (879h51m26.785296783s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.084701986 (879h51m26.78564687s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085026945 (879h51m26.78597183s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085353417 (879h51m26.786298303s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085650906 (879h51m26.786595801s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.085949286 (879h51m26.786894181s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086195117 (879h51m26.787140012s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.086467669 (879h51m26.787412564s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Uploading piece... Dec 3 03:05:00.164815383 (879h51m26.865760268s) [upload=774178]: Next returned nil segment Dec 3 03:05:00.164840239 (879h51m26.865785125s) [upload=774178]: Waiting for error group managing segments... Dec 3 03:05:00.174108452 (879h51m26.875053357s) [noise=true , address=77.90.31.15:4705 , piece_id=3PAE3AI74Y2URW5FWQLHVWS3GHAZRNXS5QXVEPLYEWRIZPXVRWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.174142326 (879h51m26.87508723s) [noise=true , address=45.91.116.13:4165 , piece_id=H36AJQ6TM42ZWYJ5NQPSSIOC6XYWDCACCYIPPYBRAVYBZACCYIVA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.175189781 (879h51m26.876134687s) [noise=true , address=77.90.31.12:4618 , piece_id=TUIE7COH6746VZAEK5CH7A4CLFLSQZHE3FQLXGOUIAL6P2LJRNYQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.178871150 (879h51m26.879816035s) [noise=true , address=145.79.227.18:4025 , piece_id=7NACG5YCV33VISNW4JX6AHLMZHLI456NQY4EVO6YTQ32MHI46TTA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.182466616 (879h51m26.883411501s) [noise=true , address=45.91.116.11:4148 , piece_id=QE6B7FVHLWTJ6XJFXAKIGWAIZATSHPOW4CA4E5UPJBSDJAJSPSJA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.184850901 (879h51m26.885795796s) [noise=true , address=45.91.116.14:4218 , piece_id=KDXUEVSB4UCJQQD4QLHOXD3RLKS2KW237S44S43K7AHHC3KV6DQQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.186313426 (879h51m26.887258321s) [noise=true , address=45.91.116.7:4039 , piece_id=HXDSSB5O467DEMIMCXZ4OWGIV5OKB4TTDZSXSD7PZYWMYDYORVPA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.187108287 (879h51m26.888053172s) [noise=true , address=45.91.116.6:4997 , piece_id=XYO3QD6DOBNHMUYYUNO2VKRPL7OMCCIA4DNLOFOMGZDDZAQLYNDQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.188257253 (879h51m26.889202148s) [noise=true , address=45.91.116.9:4288 , piece_id=5PXKYPZY7PN2TU6IXOPASYSV7YOOX7AMVRELMFN25T2XSHPX2C7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.191179818 (879h51m26.892124703s) [noise=true , address=77.90.31.18:4837 , piece_id=XQGP57SUAR7DNHY6UYF6TPCPN7MM6WMUX4WMJJCCF5BBBEJYIQ7A, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.193701038 (879h51m26.894645933s) [noise=true , address=45.91.116.5:4319 , piece_id=7V3JWIGKHHVQRHBQ5VZFSHQ6ACAKRXLKRA7QPXXLDG2QSHP5ET7Q, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.194005079 (879h51m26.894949984s) [noise=true , address=145.79.227.31:4812 , piece_id=6XBCHYA3OQJU5HLGA6MXPCKRU53ZS3T7NMQUQIDEPEADZICY2CDA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.199219063 (879h51m26.900163958s) [noise=true , address=77.90.31.13:4657 , piece_id=XOOB4JBYVH64ZTWXC4WL53VHYV5H6D5VJSYY2EQZMMYZLBDESLBQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208591191 (879h51m26.909536076s) [noise=true , address=77.90.31.11:4582 , piece_id=7NGIKVWARZJ32OLOHJNZ4L7XDV564CDJUWMWENT7XQM43BUHV4BA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: <nil> Dec 3 03:05:00.208619464 (879h51m26.909564349s) [seg_pos={0 0}, upload=774178]: Segment reached optimal threshold of 14 pieces. Dec 3 03:05:00.208754597 (879h51m26.909699482s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Newezba9KBtr5PcBVDypYdAxRFdsMtVMUTMcPWLWPLziaJhyK): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208763324 (879h51m26.909708209s) [noise=true , address=145.79.227.15:4891 , piece_id=FD4FK6VIVG22HXG7Y5B7TU2LPB3WG574TMW426N2WVQVJ3I433HQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208776869 (879h51m26.909721754s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208784383 (879h51m26.909729268s) [noise=true , address=77.90.31.19:4860 , piece_id=K5STJFARBSLBH4SLDBF6FWBMS4TBLMNS6RLHQTETJVYPTG3RXWWQ, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208884902 (879h51m26.909829797s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Dec 3 03:05:00.208890072 (879h51m26.909834967s) [noise=true , address=77.90.31.14:4676 , piece_id=GJICEDQC4NRRCFYNRY7SK43DC4FLFFN2EPIMNH6EO4GTX4EYOTNA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208952639 (879h51m26.909897524s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UhZuvUKofduRzgCWi3WQcfZVuekxBJUXydn3H1ktQ7AmRRvcx): protocol: read tcp 5.161.89.252:62308->64.49.13.154:4932: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 3 03:05:00.208961976 (879h51m26.909906891s) [noise=true , address=64.49.13.154:4932 , piece_id=SEFZ2SSUNFLGV6KXBFUANOHS4ESFEL7G67B4MWFF23O7V4SQ5IFA, seg_pos={0 0}, upload=774178]: Long tail cancellation due to optimal threshold Dec 3 03:05:00.208979329 (879h51m26.909924214s) [seg_pos={0 0}, upload=774178]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 3 03:05:00.209054830 (879h51m26.909999705s) [upload=774178]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 3 03:05:00.209061283 (879h51m26.910006168s) [upload=774178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 3 03:05:00.243054974 (879h51m26.943999869s) [upload=774178]: Done uploading segments... Dec 3 03:05:00.243070513 (879h51m26.944015398s) [upload=774178]: Upload finished. err: <nil> Duration: 423.001772ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration: 450.096232ms FAIL (2025-12-03 03:09:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 450.11111ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)